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ABSTRACT 


This  documentary  analysis  consisted  basically  of  an  evaluative 
survey  of  experimental  reports  concerning  the  field  of  programmed 
instruction.  The  purpose  was  to  identify  and  evaluate,  as  objectively 
as  possible,  the  answers,  in  terms  of  research  findings,  to  a  set  of 
questions  considered  most  important  to  the  utilization  of  programmed 
instruction.  The  method  involved  a  modified  version  of  a  technique 
formerly  utilized  by  Monroe  and  Engelhart  in  evaluating  research  done 
in  mathematics.  To  make  the  results  of  the  study  more  suitable  for 
administrative  purposes,  the  modified  version  included  a  list  of 
eight  criteria  compared  to  five  used  by  Monroe  and  Engelhart.  The 
criteria  involved  the  following  aspects  of  research:  (1)  definition 
or  explanation  of  terms,  (2)  size  of  samples  used,  (3)  duration  of 
the  experiments,  (4)  pretesting  relevant  to  the  subject  matter, 

(4)  reliability  of  the  instruments,  (5)  error  rates  of  the  programs, 

(6)  equating  of  groups,  and  (8)  statistical  treatment  of  data.  The 
criteria  were  applied  to  two  hundred  two  experimental  research  reports, 
a  process  which  yielded  an  evaluative  reliance  index  for  each  indivi¬ 
dual  report.  The  results  of  the  individual  evaluations  were  then 
applied  to  specific  findings  identified  by  the  detailed  analysis. 

This  application  in  form  of  a  dependability  level  formula  served  to 
assess  the  findings  as  being  or  not  being  highly  dependable  in  assis¬ 
ting  the  formulation  of  decisions  concerning  the  utilization  of  pro¬ 


grammed  instruction. 


The  evaluation  of  the  individual  studies  revealed  that  the 


explanation  of  terms,  the  error  rate  of  the  programs  and  the 
reliability  of  the  instruments  were  the  least  satisfied  items  of 
the  criterion  list.  The  equating  of  groups  and  the  statistical 
analysis  of  data  were  most  adequately  represented. 

Of  the  many  findings  identified  as  answers  to  the  questions 
posed  in  the  study,  relatively  few  were  found  to  be  highly  depend¬ 
able  in  terms  of  the  standard  set  by  the  dependability  level 
formula.  Programmed  instruction  was  found  to  be  at  least  as 
effective  as  conventional  methods  in  teaching  the  recognition  of 
words  in  spelling  and  reading  to  retarded  students.  Programmed 
instruction  and  conventional  methods  were  found  to  be  equally  effec¬ 
tive  in  teaching  certain  aspects  of  mathematics  to  mixed  ability 
groups  at  the  junior  high  school  level.  There  appeared  to  be  no 
difference  in  the  effectiveness  of  the  two  methods  in  teaching  most 
aspects  of  mathematics  at  the  college  level.  The  greatest  number  of 
highly  dependable  findings  were  revealed  in  English  at  all  levels  of 
achievement  and  ability  where  the  traditional  methods  and  programmed 
instruction  were  found  to  be  equally  effective  in  most  cases. 

With  respect  to  the  program  characteristics,  there  appeared 
very  little  difference  in  the  effectiveness  of  the  various  character¬ 
istics  in  achieving  the  different  educational  objectives. 
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Finally,  the  highly  dependable  findings  supported  the  conclusion 
that  programmed  instruction  did  not  serve  to  reduce  the  dependence  of 
the  learner  upon  his  ability  to  learn. 
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CHAPTER  I 


THE  INTRODUCTION 

Much  has  been  said  and  done  in  the  field  of  programmed  instruc¬ 
tion  during  the  1960 's.  Just  as  any  other  new  instructional  technique 
making  its  debut  in  the  world  of  education,  it  has  had  its  share  of 
adherents,  skeptics  and  its  quota  of  those  who  are  impartial  or 
uninformed.  The  great  divergence  of  opinion  suggests  that  the  atti¬ 
tudes  in  many  cases  may  have  been  formed  through  a  very  limited  amount 
of  direct  or  indirect  experience  with  this  particular  medium  of  instruc¬ 
tion. 


I.  THE  HISTORY  OF  PROGRAMMED  INSTRUCTION 

It  is  very  difficult  to  be  specific  in  discussing  the  origin  of 
programmed  instruction.  Literature  takes  us  as  far  back  as  400  B.C.  to 
point  out  the  famed  pedagogical  elicitations  of  the  Athenian  marvel, 
Socrates.  Plato's  "The  Meno"  is  replete  with  the  nature  of  Socrate's 
method. 1  In  an  address  at  the  Mid-Hudson  School  Board  Institute  Meeting, 
Komoski,  President  of  the  Center  for  Programmed  Instruction,  emphasized 
this  situation  by  referring  to  a  period  almost  two  thousand  years  ago 
and  reviewing  the  work  of  Quintillian  whose  words  of  advice  were: 

Do  not  neglect  the  individual  student.  He  should  be  questioned 
and  praised  ...  he  should  strive  for  victory,  yes,  but  it  must  be 


■''Carnegie  Corporation  of  New  York,  "Not  from  Teaching  but  from 
Questioning,"  Carnegie  Corporation  of  New  York  Quarterly,  Vol.  9,  No.  4 
(New  York:  Carnegie  Corporation  of  New  York,  October,  1961),  p.  1-2. 


' 
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arranged  that  he  gains  it.  In  this  way  let  us  draw  forth  his  powers 
with  both  praise  and  rewards. 2 

In  the  seventeenth  century  Rene  Descartes  enunciated  views  from  which  has 
been  borrowed  the  term  "Cartesian  Method."  This  method  was  applied  to 
program  construction  for  the  purpose  of  dividing  up  a  body  of  knowledge 
into  a  series  of  highly  presentable  forms.  Descartes  insisted  that 
nothing  be  accepted  until  its  truth  is  seen  clearly  and  distinctly.  To 
do  this  he  advised  that: 

We  must  divide  any  difficult  problem  into  smaller  and  smaller 
parts  until  we  come  to  some  proposition  so  simple  that  we  see  its 
self-evident  truth.  We  can  then  build  on  this  sure  basis,  always 
proceeding  by  small,  self-evident  steps. 3 

About  thirty-four  years  ago,  Frank  Laubach,  an  American  missionary, 
preacher  and  educator,  won  world  fame  for  teaching  millions  of  illiter¬ 
ates  how  to  read.  His  methods  were  very  similar  to  those  promulgated  by 
enthusiastic  supporters  of  programmed  instruction  today.  Mr.  Laubach  in 
describing  the  techniques  presented  in  his  autobiography  states  that: 

We  must  prove  to  the  student  that  he  can  learn  easily,  quickly, 
and  delightfully,  no  matter  how  old  he  is.  Every  step  must  be  so 
short  that  any  ordinary  man  can  take  it  easily.  Our  charts  must 
provide  for  this  ...  for  there  must  be  no  embarrassing  pauses, 
never  a  question  the  student  cannot  answer,  and  no  examination  to 
find  out  what  he  knows  ...  We  must  keep  out  of  the  student's  way-- 
neither  pushing  nor  retarding  him.  For  a  student  is  happy  only 
when  he  feels  free  to  take  his  natural  gait  ...  The  ordinary  text¬ 
books  used  in  the  schools  of  America  require  constant  talking  on 
the  part  of  the  teacher  but  our  ...  texts  are  as  nearly  self - teaching 


2 

P.  Kenneth  Komoski,  "Call  It  Programmed  Instruction  Not  a 
Teaching  Machine,"  Mid-Hudson  Channel ,  10:9,  January,  1961. 

3 

W.  T.  Jones,  "Rene  Descartes,"  The  World  Book  Encyclopedia 
(1962  ed.) ,  Volume  4,  130. 
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as  we  can  make  them.  They  require  little  talking  by  the  teacher  and 
a  maximum  of  participation  by  the  student. 4- 

Mr.  Laubach's  recognition  of  the  individuality  of  the  learner  is  further 

illustrated  with  a  single  condensation  of  his  principles  of  teaching: 

"Each  one  teach  one."^ 

In  1926  and  1927  some  of  the  principles  of  programmed  instruction 
were  implied  in  Pressey's  description  of  a  simple  teaching  machine.^ 

The  purpose  of  Pressey's  teaching  apparatus  was  to  produce  efficient 
practice,  a  basis  quite  in  keeping  with  Thorndike's  "Law  of  Effect" 
and  "Law  of  Exercise"  which  constituted  an  important  part  of  the  learning 
theory  of  that  day.  Pressey's  subsequent  articles^  included  predictions 
of  an  impending  revolution  in  education;  however,  his  prophecies  did  not 
gain  widespread  recognition  until  Skinner's  arrival  on  the  scene  in  1954. 
Although  Pressey  failed  to  make  an  adequate  impression  upon  the  educators 
of  that  day,  his  pioneering  did  serve  to  inspire  his  students  into 


P.  Kenneth  Komoski,  "Programmed  Instruction  and  Its  Place  in 
Education"  (an  address  presented  at  the  Annual  Educational  Conference 
sponsored  by  the  Educational  Records  Bureau,  New  York,  N.Y.,  1960). 

^Galen  Saylor,  "Frank  Charles  Laubach,"  The  World  Book  Encyclo¬ 
pedia  (1962  ed.) ,  Volume  11,  111. 

£ 

DSydney  L.  Pressey,  "A  Simple  Apparatus  Which  Gives  Tests  and 
Scores  -  and  Teaches,"  School  and  Society,  23:373-376,  March,  1926; 
Sydney  L.  Pressey,  "A  Machine  for  Automatic  Teaching  of  Drill  Material," 
School  and  Society,  24:549-552,  1927. 


7 

Sydney  L.  Pressey,  "A  Third  and  Fourth  Contribution  Towards  the 
Coming  'Industrial  Revolution'  of  Education,"  School  and  Society,  36:668- 
672,  November,  1932;  Sydney  L.  Pressey,  "Development  and  Appraisal  of 
Devices  Providing  Immediate  Automatic  Scoring  of  Objective  Tests  and  Con¬ 
comitant  Self-Instruction,"  Journal  of  Psychology,  29:417-447,  February, 
1950. 
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producing  a  considerable  amount  of  classroom  research.  In  addition,  his 
work  was  very  influential  in  the  development  of  the  subject  matter 
training  programs  for  the  United  States  Air  Force. 

On  March  12,  1954,  Professor  B.  F.  Skinner  of  Harvard  University 
presented  a  penetrating  account  of  programmed  instruction  before  a  con¬ 
ference  on  Current  Trends  in  Psychology  and  Behaviour  Sciences  which  was 
held  at  the  University  of  Pittsburgh.®  The  thesis  of  Skinner's  paper  held 
that  a  teacher  alone  was  an  inefficient  and  outmoded  instrument  in  the 
application  of  the  psychological  theory  of  learning  to  the  process  of 
instruction.  It  is  difficult  to  determine  whether  it  was  Skinner's 
assertion  that  teachers  were  relatively  inefficient  instruments  or  whether 
it  was  the  steadily  increasing  need  for  more  education  created  by  the 
tremendous  growth  in  population  and  industry  that  made  programmed  instruc¬ 
tion  so  appealing  to  a  great  many  educators.  Perhaps  both  factors  were 
instrumental. 

Programmed  instruction  as  originally  conceived  was  based  on  a 
learning  theory  related  to  the  research  done  with  animals.  Its  basis  as 
outlined  by  some  authorities  naturally  inherited  a  rigidity  devoted  to 
certain  elements  of  behaviourist  psychology.  In  fact  it  would  be  quite 
safe  to  conclude  that  over  ninety  percent  of  the  programs  written  to 
date  are  of  the  Skinnerian  type.  The  introduction  of  Crowder's  branched 
programming  technique  was  an  effort  to  move  away  from  Skinner's  stereo¬ 
type  in  order  to  utilize  the  principles  basic  to  other  existing  theory 

®B .  F.  Skinner,  "The  Science  of  Learning  and  the  Art  of  Teaching," 
The  Harvard  Educational  Review,  24:86-97,  Spring,  1954. 
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of  learning. 

A  more  recent  development  in  the  movement  away  from  the  linear  or 
Skinnerian  format  has  been  the  technique  of  mathetical  programming. 
Conceived  by  Dr.  Gilbert  in  the  late  1950's,  mathetics  is  regarded  as  a 
decided  advance  in  the  technology  of  programmed  instruction.^  Mathetical 
programming  is  based  on  systematic  subject  analysis  rather  than  the  rigid 
format  so  typical  of  programming  methods  today.  In  describing  mathetical 
programs  Reynolds  states: 

Even  so,  examination  of  most  mathetical  lessons  to  date  reveals  a 
common  format:  one  that  resembles  conventional  workbooks,  rather 
than  linear  or  branching  programs.  The  systematic  method  of  presen¬ 
tation  used  in  mathetical  lessons  is  referred  to  as  the  'exercise 
model.'  Subject  matter  is  presented  to  the  student  at  three  different 
levels  of  difficulty.  Each  level  is  represented  by  an  exercise  which 
may  vary  in  size  from  less  than  a  page  to  several  pages.  Achieving 
the  maximum  amount  of  instruction  per  unit  is  the  factor  that  deter¬ 
mines  exercise  length.  Within  the  framework  of  the  exercise  model, 
mathetical  lessons  also  use  branching  to  make  allowances  for  differ¬ 
ences  in  the  abilities  of  students. 10 

In  concluding,  then,  it  seems  reasonable  to  expect  that  future 
developments  will  consist  largely  of  broadening  the  varieties  of  program¬ 
med  techniques  in  an  effort  to  accommodate  specific  learning  objectives. 
One  may  also  conclude  that  although  programming  in  a  certain  sense  has 
always  been  with  us,  its  recognition  did  not  achieve  intensity  until  its 
effectiveness  became  the  subject  of  very  deliberate  scrutiny  through 
experimentation. 


9 

Robert  L.  Reynolds,  "A  Teaching  Tool  -  for  the  Health  Professions,' 
NSPI  Journal,  2:7,  December,  1963. 


10 


Ibid. 
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II.  THE  PROBLEM 

Numerous  questions  have  been  asked  and  more  are  being  asked 
about  the  efficacy  of  programmed  ins truction. 1 1  Since  there  is  no 
point  in  education  at  which  the  evidence  concerning  some  instructional 
problem  is  complete,  it  behooves  the  educator  to  derive  answers  based 
on  the  information  that  is  available.  It  is  possible,  of  course,  for 
an  educator  to  arrive  at  some  specific  conclusions  by  means  of  his 
own  experimental  work;  however,  it  is  not  likely  that  a  great  deal 
will  be  accomplished  with  respect  to  the  field  as  a  whole.  Further¬ 
more,  many  people  concerned  with  administering  education  generally  do 
not  have  the  time,  knowledge,  nor  the  facilities  to  conduct  experi¬ 
mental  investigations;  consequently,  they  must  resort  to  the  applica¬ 
tion  of  the  research  work  completed  by  others.  Since  the  writer 
believes  that  very  careful  consideration  should  be  given  to  the  type 
of  research  which  is  to  be  utilized  in  administrative  guidance,  this 
study  was  designed  to  investigate  the  research  done  by  others  in  order 


^John  0.  Cook  and  Howard  G.  Miller,  "Programmed  Learning," 

A  Guide  to  Curriculum  Study  (Raleigh,  North  Carolina:  Department  of 
Curriculum  Study  and  Research,  State  Board  of  Education,  1962) , 
pp.  40-50;  George  E.  Probst  (ed.) ,  Programmed  Learning  in  the  Schools: 
Tasks  for  1962 .  A  Summary  of  a  Conference  held  by  the  Thomas  Alva 
Edison  Foundation  and  Grolier  Incorporated,  Albuquerque,  New  Mexico, 
January  13-14,  1962  (New  York:  Thomas  Alva  Edison  Foundation,  Inc., 
1962), pp.  23-24;  John  D.  Krumbholtz,  "Needed  Research  in  Programmed 
Instruction,"  Educational  Leadership ,  October,  1963,  pp .  32-33; 

Ruth  L.  Godwin,  "Do  We  Understand  the  Place  of  Programmed  Instruction 
in  the  Teaching  Complex?"  The  ATA  Magazine ,  43:21-25,  March,  1963; 

Allen  Jack  Edwards,  "Learning  Theory  and  Programmed  Learning:  Problems 
of  Implementation,"  The  University  of  Kansas  Conference  on  Programmed 
Learning,  Kansas  Studies  in  Education,  Vol.  II,  No.  2  (Lawrence, 

Kansas:  The  University  of  Kansas  Publications,  School  of  Education, 1961) , 
pp.  15-21. 
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to  determine  the  nature  of  the  answers  to  some  important  questions 
basic  to  the  field  of  programmed  instruction.  Specifically,  it  was 
the  purpose  of  this  study  to  gather,  to  analyze  and  to  evaluate  the 
pertinent  scientific  research  reports  which  were  available  to  the 
writer  through  the  various  media  listed  elsewhere  in  the  study.  It 
was  also  the  purpose  of  this  study  to  apply  this  evaluation  in  as  an 
objective  manner  as  possible  to  the  generalizations  discovered  and  to 
express  their  utility  in  terms  of  a  dependability  level  suitable  to  be 
considered  in  the  administration  of  education. 

III.  ELEMENTS  OF  THE  PROBLEM 

The  first  element  consisted  of  the  evaluation  of  the  available 

research  reports  in  terms  of  an  established  evaluative  criteria  derived 

from  the  fundamentals  of  adequate  research  reporting  similar  to  the 

1  o 

criteria  developed  by  Monroe  and  Engelhart. 

The  second  element  involved  the  determination  of  answers,  and 
the  evaluation  of  these  same  answers  to  the  questions  posed  in  the 
study.  The  answers  were  determined  by  referring  specifically  to  the 
findings  of  the  studies  that  were  analyzed.  The  evaluation  consisted 
of  the  application  of  the  results  obtained  in  element  one. 


■^Walter  S.  Monroe  and  Max  Engelhart,  A  Critical  Summary  of 
Research  Relating  to  the  Teaching  of  Arithmetic ,  Bulletin  No.  58 
(Urbana,  Illinois:  University  of  Illinois  Press,  1931). 


. 
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IV.  DELIMITATIONS  OF  THE  PROBLEM 

Basically,  the  problem  investigated  by  this  study  was  delimited 
in  two  ways:  (1)  by  the  characteristics  of  the  sources  of  data  used, 
and  (2)  by  the  questions  found  to  be  most  frequently  asked  by  educators 
Characteristics  of  the  Sources  of  Data 

1.  Generally,  only  investigations  which  were  supported  by 

universities,  professional  organizations  and  institutions, 
government  agencies  and  educational  systems  were  included 


in  this  documentary  analysis. 

2.  Only  reports  of  experimental  investigations  specifically 
concerned  with  some  relevant  aspect  of  programmed  instruc¬ 
tion  were  analyzed. 

3.  Only  those  research  reports  contributing  to  the  formulation 
of  conclusions  pertaining  to  the  questions  posed  in  this 
investigation  were  considered. 

Questions  Which  Delimited  the  Prob lem 

1.  Can  programmed  instruction  be  used  successfully  in  excep¬ 
tional  cases?  Exceptional  circumstances  were  meant  to  imply 
conditions  which  ordinarily  would  not  be  handled  in  a 
"typical"  classroom  manner. 

2.  In  which  subject  areas  can  programmed  instruction  serve 
successfully? 

3.  Can  programmed  instruction  be  used  at  any  grade,  ability  or 
age  level? 
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4.  How  effective  is  programmed  instruction  compared  to  tradi¬ 
tional  instruction? 

5.  How  effective  is  programmed  instruction  compared  to  other 
instructional  materials? 

6.  Which  are  the  more  effective  and  efficient  means  of  presen¬ 
ting  programmed  materials? 

7.  Is  the  effectiveness  and  the  efficiency  of  the  programmed 
instructional  materials  dependent  upon  the  type  of  response 
mode  used? 

8.  Is  feedback  a  necessary  process  in  programmed  instruction? 

V.  LIMITATIONS  OF  THE  STUDY 

The  study  was  affected  by  both  expected  and  unexpected  limita¬ 
tions.  The  following  list  indicates  the  limitations  of  this  investi¬ 
gation: 

1.  The  study  was  limited  by  the  success  in  obtaining  suitable 
literature  which  reported  experimental  projects  in  the 
field  of  programmed  instruction. 

2.  The  study  was  limited  by  the  amount  of  time  which  was 
available  in  which  to  collect  and  analyze  the  data. 

3.  The  interpretation  of  the  conclusions  was  restricted  to 
the  basic  limiting  assumptions  which  had  to  be  made  before 
a  study  of  this  nature  could  be  undertaken. 
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VI.  LIMITING  ASSUMPTIONS 

The  nature  of  this  evaluative  study  required  a  statement  of 
three  basic  assumptions  of  the  limiting  type  described  by  Rummel.^ 

1.  It  was  assumed  in  this  study  that  all  evaluative  criterion 
items  were  equally  important  in  assessing  the  findings  from 
the  utilitarian  point  of  view  in  educational  administration. 
A  satisfied  criterion  item  was  given  a  relative  value  of 
one . 

2.  With  respect  to  the  findings  of  the  individual  studies,  it 
was  assumed  that  a  reliance  index  of  8  represented  a  high 
reliance  and  that  any  smaller  number  represented  a  low 
reliance . 

3.  It  was  assumed  that  the  ratio  of  0.9  or  higher  as  deter¬ 
mined  through  the  application  of  the  dependability  level 
formula  represented  a  high  dependability  level  of  the 
findings  supported  by  one  or  a  group  of  studies.  A  ratio 
below  0.9  indicated  a  low  dependability  level. 

VII.  TERMINOLOGY  OF  PROGRAMMED  INSTRUCTION 

As  already  pointed  out,  the  principles  upon  which  programming 
is  based  are  not  entirely  new.  However,  it  is  only  recently  that  pro¬ 
grammed  instruction  has  gained  a  rather  widespread  recognition  as  a 

Francis  Rummel,  An  Introduction  to  Research  Procedures  in 
Education  (New  York:  Harper  &  Brothers,  1958),  pp.  129-130. 
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likely  potential  in  the  more  expeditious  application  of  the  laws  of 
learning.  Glaser  refers  to  programmed  learning  as  a  sort  of  applied 
psychology.!^  Today's  critics  are  guilty  of  rigidly  associating  the 
concepts  of  programmed  instruction  with  the  beliefs  and  findings  of 
Pressey,  Skinner  and  Crowder.  Mager,  in  describing  a  method  for 
preparing  auto-instructional  programs  implies  that  this  should  not  be 
so  by  pointing  out  that: 

Other  programmers  approach  the  problem  differently  and  assign 
greater  or  lesser  weight  to  the  steps  outlined  below. 15 

He  goes  on  to  emphasize  that: 

--  a  good  program  is  one  that  works  rather  than  one  which  con¬ 
forms  to  some  writing  style  or  programming  strategy. 16 

It  appears  that  there  is  a  need  for  a  more  specific  definition  that 

is  all  embracing  with  respect  to  the  theory  of  learning.  This  section 

of  the  study  was  devoted  to  the  presentation  of  the  programming  jargon 

and  to  the  demonstration  of  its  flexibility  concerning  the  theory  of 

learning.  To  accomplish  this,  each  term  was  accompanied  by  the 

relevant  theory  whenever  such  an  arrangement  seemed  appropriate. 

Several  references  were  consulted  in  discussing  the  definitions  of 

the  terms  included  in  this  section. 


"^Robert  Glaser,  "Programmed  Instruction  -  A  Behavioral  View,1 
The  American  Behavioral  Scientist,  November,  1962,  p.  46. 


Programs , " 
1961. 


F.  Mager,  "A  Method  for  Preparing  Auto-Instructional 
A  Reprint,  p.  1.  Institute  of  Radio  Engineers,  December, 


16 


Ibid. 
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Programmed  instruction.  Previous  definitions  were  identified 
rather  closely  with  the  Skinnerian  concept  of  the  learning  process 
which  depicted  programmed  instruction  as  a  technique  characterized  by 
a  rigidly  controlled  presentation  of  material,  the  elicitation  of  an 


appropriate  response,  guidance  with  respect  to  the  subject  matter  and 
control  of  the  way  in  which  learning  proceeds.  Hilgard  justifies  the 
acceptance  of  Skinner's  prototype  on  the  premises  that: 


Programmed  instruction  recognizes  individual  differences  by 
beginning  where  the  learner  is  and  by  permitting  him  to  work  at 
his  own  pace;  programmed  instruction  requires  the  learner  to  be 
active;  knowledge  of  results  are  available  immediately;  there  is 
a  continuity  between  the  easier  and  the  harder  concepts  in  the 
organization  of  the  program;  high  degree  of  success  is  guaranteed 
by  spaced  review;  programmed  instruction  reduces  anxiety  because 
the  learner  is  not  threatened  by  the  task. 17 


Schramm  argues  that  the  Skinnerian  concept  obviously  does  not  include 
psychological  theory  other  than  Skinner's  operant  conditioning, 

I  O 

Pavlovian  conditioning  or  Guthrie's  contiguity  learning.  °  Pertaining 
to  the  various  existing  psychological  theories  of  learning,  recent 
trends  in  definition  appear  to  be  more  flexible.  Cook  and  Miller  define 


programmed  instruction  as  being: 

...  simply  a  rational,  experimental  and  testable  means  of 
achieving  whatever  goals  are  desired. 19 


■^Ernest  R.  Hilgard,  "What  Support  from  the  Psychology  of 
Learning,"  NEA  Journal ,  50:20-21,  November,  1961. 

■^Wilbur  Schramm,  Programmed  Instruction  Today  and  Tomorrow 
(New  York:  The  Fund  for  the  Advancement  of  Education,  1962),  p.  3. 

l^John  0.  Cook  and  Howard  G.  Miller,  "Programmed  Learning,"  A 
Guide  to  Curriculum  Study  (Raleigh,  North  Carolina:  Department  of  Cur¬ 
riculum  Study  and  Research,  State  Board  of  Education,  1962),  p.  3. 
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Kopstein  and  Shillestad  consider  the  term  programmed  instruction  to  be 
synonymous  with  auto- ins truction  and  state  that: 

From  the  scientific,  reductionist  point  of  view  auto- ins truction 
is  nothing  but  an  application  of  the  principles  governing  the 
interaction  between  the  teaching  presentation  and  the  learning 
individual . 20 

In  elaborating  they  add  that: 

It  is,  however,  a  minutely  systematic  application  of  the  scien¬ 
tific  principles  of  learning;  and  the  application  itself  is  guided 
by  the  general  principles  of  engineering  design  logic. 21 

In  a  recent  issue  of  the  NSPI  Journal,  Robert  G.  Smith,  Presi¬ 
dent  of  the  National  Society  for  Programmed  Instruction,  suggested 
that  programmed  instruction  be  defined  in  terms  of  the  process  of 
program  preparation.  This  process  would  involve: 

1.  Valid  and  clearly  defined  objectives,  which  represent 
statements  of  the  behaviour  desired  by  the  student,  are  developed. 

2.  A  criterion  test  to  evaluate  the  attainment  of  these  objec¬ 
tives  is  prepared. 

3.  The  content  needed  by  the  student  so  he  can  acquire  the 
necessary  behaviour  is  embedded  in  the  appropriate  media,  applying 
known  principles  of  instruction.  This  content  is  a  program. 

4.  The  program  is  tested  by  giving  students  the  criterion  test 
and  revised  and  retested  until  the  students  reach  a  high  level  of 
mastery  of  the  ob jec tives . 22 


20Ibid. 

2 1 

Felix  F.  Kopstein  and  Isabel  J.  Shillestad,  A  Survey  of  Auto- 
Instructional  Devices ,  ASD  Technical  Report  61-414  (Wr ight-Patter son 
Air  Force  Base,  Ohio:  Aeronautical  Systems  Command,  United  States 
Air  Force,  1961),  p.  33. 

22Robert  G.  Smith,  "President's  Insights,"  NSPI  Journal,  2:15, 
October,  1963. 
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In  concluding,  it  seems  that  programmed  instruction  is  intended 
to  be  something  more  than  just  a  standardized  method  of  presenting 
information  to  the  student  and  making  him  respond  to  it  in  a  desirable 
fashion.  Its  potentiality  as  an  effective  instructional  technique  is 
implied  to  be  contingent  on  its  flexibility  to  incorporate  the  various 
principles  of  learning.  Since  it  includes  a  precise,  systematic 
analysis  of  behaviour  reflecting  subject-matter  competence,  an  assess¬ 
ment  of  the  objectives  of  learning  and  a  very  careful  testing  in  the 
field  or  laboratory,  one  might  be  quite  justified  in  referring  to  it 
as  "validated  instruction." 

Linear  program.  This  is  a  type  of  program  in  which  the  sequence 
of  items  is  fixed,  unalterable  and  identical  for  each  sequence.  Some¬ 
times  this  program  is  referred  to  as  extrinsic  because  the  rate  and 
sequence  of  presentation  are  determined  by  the  person  writing  the 
program.  Because  the  sequence  is  fixed,  this  type  of  program  is  also 
called  straight  line,  non-branching  or  Skinnerian.  The  learning  theory 
forming  the  basis  of  this  type  of  program  is  generally  thought  to 
include  borrowed  portions  from  the  behavioural  psychologists  sometimes 
referred  to  as  the  associationists  or  members  of  the  stimulus-response 
school.  The  development  of  this  amalgam  is  highlighted  by  investiga¬ 
tions  of  such  noted  personalities  as  Ebbinghaus,  Pavlov,  Thorndike, 
Pressey,  Witaaek,  Gates,  Guthrie  and  Skinner.  Skinner  may  be  consi¬ 
dered  as  representing  all  since  he  was  responsible  for  synthesizing 
the  relevant  portions  of  the  S-R  theory  promulgated  by  his  predecessors 
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and  applying  it  to  a  learning  technique  which  involved  active  parti¬ 
cipation  on  the  part  of  the  learner,  presentation  in  small  steps,  an 
abundance  of  repetition,  immediate  reinforcement  of  the  response, 
knowledge  of  the  results  and  maximum  reduction  of  incorrect  responses. 
The  work  of  Witaaek  and  Gates  is  represented  by  the  fact  that  the 
learner  must  be  active.  Skinner's  shaping  theory  is  implied  when 
learning  takes  place  in  small  steps.  Sufficient  repetition  to  insure 
overlearning  represents  the  psychology  of  Ebbinghaus  and  Pavlov. 
Thorndike's  "Law  of  Effect"  is  operant  in  the  immediate  reinforcement 
of  the  correct  response.  Providing  for  conditions  which  will  mini¬ 
mize  the  number  of  incorrect  responses  utilizes  Pressey's  "Law  of 
Recency."  It  is  interesting  to  note  that  the  majority  of  the  programs 
being  produced  at  present  are  linear  in  format. 

Branched  program.  J  Conceived  by  Norman  Crowder,  this  type 
of  program  has  a  built-in  alternate  sequence  of  items  in  order  to 
render  it  adaptive  to  the  varying  capabilities  of  the  learner.  The 
kind  of  response  which  a  learner  makes  determines  the  sequence 
through  which  he  will  be  guided.  Each  sequence  proceeds  methodically 
toward  a  given  learning  objective.  The  more  knowledgeable  student  is 
permitted  to  omit  certain  frames,  while  the  less  knowledgeable  is 
required  to  complete  additional  frames.  Unlike  the  linear  program, 
there  is  no  assumption  made  in  the  branched  program  that  the  wrong 

^David  Cram,  Explaining  "Teaching  Machines"  and  Programming 
(San  Francisco:  Fearon  Publishers,  Inc.,  1961),  p.  41. 
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answer  hinders  the  learning  of  the  correct  response.  Rather,  the 
purpose  of  the  response  is  to  guide  the  learner  through  the  program. 
According  to  Crowder,  the  response  is  a  measuring  and  a  guiding  device 
rather  than  a  fixing  device.  The  effects  of  wrong  answers  are  presum¬ 
ably  nullified  by  the  way  feedback  is  presented.  The  student  is  not 
merely  told  that  he  is  wrong  but  also  informed  as  to  why  he  is  wrong. 

By  providing  such  feedback,  this  program  satisfies  the  psychological 
theory  that  the  learner  should  work  at  a  level  near  to  his  capacity. 

The  program  also  conforms  to  Woodworth's  judicious  statement  that: 

In  a  practical  situation  it  is  probably  best  to  start  with  the 
whole  method  while  feeling  free  to  concentrate  on  a  part  where 
something  special  is  to  be  learned. 24 

Eclectic  program.  An  eclectic  program  is  one  which  has  incor¬ 
porated  the  more  desirable  features  of  both  the  linear  and  the  branched 
formats.  Logically,  it  should  be  the  most  effective  type  since  its 
design  would  be  accommodated  by  a  wider  range  of  psychological  theory. 
No  research  has  been  conducted,  however,  to  indicate  that  this  is  so. 

Covert  response .  A  covert  response  is  a  physically  non-active 
response  to  a  question  in  a  program.  It  requires  thinking  about  the 
answer . 

Cue .  This  is  a  term  used  interchangeably  with  prompt  which 
refers  to  an  additional  bit  of  information  included  in  a  program  for 

S.  Woodworth,  Experimental  Psychology  (New  York:  Holt, 
1938) ,  p.  223. 
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the  purpose  of  ensuring  the  elicitation  of  the  correct  response. 2^  its 
inclusion  in  the  program  is  justified  by  Pressey's  "Law  of  Recency" 
which  implies  that  the  last  response  is  the  one  which  is  remembered  best. 
As  a  result,  it  becomes  imperative  that  it  be  correct. 

Cue,  vanishing.  A  vanishing  cue  is  a  bit  of  information  which 
is  gradually  withdrawn  or  eliminated  entirely  as  the  learner  moves 
through  the  program. 28  it  is  reaiiy  a  part  of  the  shaping  process. 

Error .  In  programmed  instruction  the  error  is  defined  as  a 

2  7 

response  which  is  not  acceptable  to  the  programmer. 

Error  rate .  Generally,  error  rate  refers  to  the  percentage 

28 

of  incorrect  responses  on  an  item,  a  set  of  items  or  a  whole  program. 

Extrinsic  programming.  It  is  the  process  of  developing  a 
linear  program  in  which  the  sequence  of  items  determined  by  a  prog¬ 
rammer  is  unalterable.  Every  student  must  work  through  the  same 
29 

sequence . 

“^Charles  I.  Foltz,  The  World  of  Teaching  Machines  (Washington, 
D.C.:  Electronic  Teaching  Laboratories,  1961),  p.  67. 

9  A 

°Teaching  Machine  Terms:  A  Glossary  (Bulletin  4.  Hackensack, 
New  Jersey:  Universal  Electronics  Laboratories  Corporation,  1962),  p.  2. 

27 

Iowa  Lakeside  Laboratory,  Programming  as  a  Concern  to  Education 
(Summary  Report  of  the  Eighth  Okoboji  Audiovisual  Leadership  Conference. 
Milford,  Iowa:  Iowa  Lakeside  Laboratory,  1962),  p.  64. 

28Ibid. 

2<^Charles  I.  Foltz,  The  World  of  Teaching  Machines  (Washington, 
D.C.:  Electronic  Teaching  Laboratories,  1961),  p.  67. 
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Feedback.  Feedback  is  that  technique  used  in  programming  which 

provides  the  learner  with  the  immediate  knowledge  of  the  suitability 

80 

of  his  response.  Reinforcement  of  the  correct  response  is  facili¬ 
tated  by  the  process  of  feedback. 

Frame.  A  frame  is  a  step  in  a  program.  It  contains  information 
and  a  stimulus  to  which  the  learner  responds.  A  frame  usually 

represents  a  piece  of  knowledge  in  its  most  comprehensible  form. 

Intrinsic  program.  This  is  a  type  of  program  which  uses  the 
technique  of  branching  previously  described  in  this  list  of  defini¬ 
tions  . 


I  tern.  The  term  item  is  used  interchangeably  with  the  word 

frame . 


Confirmation.  Confirmation  is  a  built-in  provision  which  serves 

as  either  a  feedback  or  a  reinforcing  function.  In  this  manner  the 

32 

learner  is  made  aware  of  his  progress  at  very  frequent  intervals. 

Novelty  Effect .  This  is  an  effect  produced  by  the  human  factor 

O  O 

which  influences  behaviour  in  a  new  situation. 

30Ibid. 

31Ibid. 

O  O 

David  J.  Klaus,  Job  Specifications  -  Programmer  Technician 
(Pittsburgh:  American  Institute  for  Research,  1963),  p.  4. 

33Charles  I.  Foltz,  The  Wor Id  of  Teaching  Machines  (Washington, 
Electronics  Teaching  Laboratories,  1961),  p.  67. 


D.  C .  : 
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Other  than  Popham's  study,  which  indicated  the  insignificance  of  the 
Novelty  Effect  in  invalidating  research  findings,  no  convincing  evidence 
has  been  produced  to  determine  its  function  in  programmed  learning. ^ 

Dallard  and  Williams  reminiscence  effect .  It  is  an  effect 
which  results  in  an  increased  performance  of  the  learner  immediately 
after  the  learning  period  has  ceased.  The  effect  is  not  enduring  but 
it  is  important  to  be  aware  of  its  possible  existence  when  designing 
experiments . 

Overt  response .  An  overt  response  refers  to  any  physical 
activity  involved  in  the  learner^  reaction  to  a  stimulus  contained 
within  a  program. 

Pace .  Pace  is  the  rate  at  which  the  student  is  permitted  to 

O  (L 

work  through  the  programmed  material.  ° 

Prompt.  A  prompt  is  a  clue  introduced  to  the  program  in 
order  to  make  responding  easier  for  the  learner.  The  term  is  used 
interchangeably  with  cue. 

Reinforcement.  Reinforcement  is  basic  to  programmed  learning 

34w.  James  Popham,  "Novelty  Effect  of  Programmed  Instruction" 

(a  paper  presented  at  the  first  annual  convention,  N.S.P.I.,  San 
Antonio,  Texas,  March  29,  1963). 

■^Foltz,  loc .  cit . 

^Teaching  Machine  Terms:  A  Glossary,  loc.  cit. 
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theory.  It  is  the  process  of  supplying  feedback  or  information  to  the 
learner  concerning  the  correctness  of  his  response. ^ 

Step .  A  step  is  an  increment  in  subject  matter  level  as  the 
learner  progresses  from  one  item  to  another. 

Shaping .  Shaping  is  the  process  of  differential  reinforcement 
of  increasingly  more  adequate  forms  of  behaviour  until  the  terminal 

O  Q 

behaviour  is  achieved. 

VIII.  DEFINITION  OF  TERMS 

The  following  terms  were  pertinent  to  the  analysis  and  evalua¬ 
tion  done  in  this  investigation: 

Efficiency.  Ef f iciency  was  the  ratio  which  represented  the 

amount  learned,  as  measured  by  a  criterion  test,  divided  by  the  length 
of  the  learning  period. 

Effectiveness .  Effectiveness  was  that  quality  of  an  instruc¬ 
tional  technique  which  produced  learning.  It  was  expressed  in  terms  of 
the  scores  obtained  on  a  criterion  test  independent  of  the  length  of 
the  learning  period. 

Q  7 

Foltz,  Toe.  ci t . 

■^Teaching  Machines :  A  Glossary,  loc.  cit. 

39Ibid. 


Evaluation.  This  term  referred  to  the  assessment  of  research 


reports  on  the  basis  of  criteria  which  were  relevant  to  this  particu¬ 
lar  study.  "Evaluation"  was  not  synonymous  with  the  quality  of 
research . 

Reliance  index.  The  reliance  index  was  the  number  of  evalua¬ 
tive  criterion  items  which  were  satisfied  by  each  report  that  was 
analyzed  in  this  study.  The  reliance  index  of  each  report  was  deter¬ 
mined  on  the  basis  of  the  presence  or  absence  of  each  specified  item 
listed  in  the  evaluative  criteria.  A  numerical  value  of  one  was 
assigned  to  each  criterion  item.  The  maximum  number  of  items  was  eight 
which  represented  the  highest  possible  reliance  index  relative  to  one. 

Dependability  level .  Dependability  level  was  that  quality  of 
a  conclusion  or  finding  which  was  determined  by  the  application  of  the 
dependability  formula  to  the  reliance  indices  that  supported  or  did  not 
support  a  conclusion  or  finding. 

Instructional  ob  jec tives .  The  instructional  objectives  included 
those  specifically  desired  outcomes  of  learning  common  to  most  content 
areas.  These  were  concept  formation,  development  of  understanding  or 
comprehension,  rote  memorization,  assimilation  of  facts,  development 
of  problem  solving  ability,  acquisition  of  generalizations  or  principle 
application  of  the  acquired  learnings,  and  transfer  of  training. 

Content  ob  jectives .  The  objectives  were  statements  indicating 
the  specific  subject  areas  in  which  the  instructional  aims  were  to  be 
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achievedo 


IX.  REVIEW  OF  RELATED  LITERATURE 

The  review  of  related  literature  was  limited  to  a  survey  of 
descriptive  reports  based  on  documentary  compilations  and  analyses. 

The  similarity  of  design  to  the  writer's  study  was  considered  as  the 
criterion  of  relationship  rather  than  the  similarity  of  content 
material  since  the  main  body  of  the  project  included  an  analysis  and 
an  evaluation  of  most  of  the  available  literature  not  of  the  survey 
type.  In  order  to  emphasize  the  similarities  and  the  differences 
between  this  study  and  others  which  utilized  the  documentary  reference 
approach,  the  review  of  related  literature  included  some  studies  out¬ 
side  the  area  of  programmed  instruction.  All  the  related  studies, 
however,  were  confined  to  the  field  of  education. 

One  of  the  first  survey  type  projects  completed  in  the  field 
of  programmed  instruction  was  a  source  book  edited  by  Lumsdaine  and 
Glaser. The  book  was  largely  a  compilation  of  research  reports 
along  with  numerous  articles  not  based  on  experimental  evidence.  It 
excluded  any  objective  attempt  at  evaluating  the  conclusions  and  pro¬ 
cedures  which  were  reported. 

Silberman  reviewed  ninety-six  articles  and  research  reports. 

His  survey  included  literature  dealing  with  the  response  mode,  com¬ 
parison  of  programmed  instruction  with  the  more  conventional  techniques, 

^A.  A.  Lumsdaine  and  Robert  Glaser  (eds.),  Teaching  Machines  and 
Programmed  Learning:  a  Source  Book  (Washington,  D.C.:  National  Educa¬ 
tion  Association,  1961) ,  310  pp. 
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trends  in  programmed  instruction  and  the  problems  involved  in  imple¬ 
menting  this  new  approach  to  learning.  Some  general  statements  were 
made;  however,  as  in  the  case  of  Lumsdaine ' s  and  Glaser's  Source  Book, 
no  effort  was  made  to  specifically  evaluate  the  contents  of  the 
literature . 

Another  of  the  more  recent  reviews  was  made  by  Wilbur  Schramm 
who  surveyed  research  concerned  with  several  aspects  of  programmed 
instruction  involving  prompting,  cueing,  confirmation,  sequence  of 
steps,  size  of  step,  branching,  reviewing,  response  modes  and  the 
feedback  process.  Again,  as  in  most  reviews  within  this  field,  the 
work  was  largely  a  compilation  of  findings  in  which  no  specific  attempt 
was  made  at  analyzing  and  evaluating  the  documents  that  were  reviewed. ^ 

Briggs  and  Angell  surveyed  research  concerning  the  programming 
of  instruction  in  mathematics  and  science.  Twenty-six  studies  which 
compared  programmed  instruction  with  the  conventional  methods  were 
included  as  part  of  the  survey.  In  evaluating  they  concluded  that: 

These  studies,  as  a  group,  are  not  characterized  by  sophisti¬ 
cated  experimental  design  nor  by  scrupulous  control  of  extraneous 
variables.  However,  the  majority  of  them  did  employ  some  form  of 
statistical  test  to  assess  the  significance  of  differences  among 
groups  on  criterion  test  scores.  Findings  of  "no  significant 
differences"  were  reported  more  often  than  findings  of  significant 


^Harry  F.  Silberman,  "Se lf-Teaching  Devices  and  Programmed 
Materials,"  Review  of  Educational  Research,  332:179-193,  April,  1962. 

^Wilbur  Schramm,  Programmed  Instruction  Today  and  Tomorrow 
(Washington,  D.C.:  Fund  for  the  Advancement  of  Education,  1962), 
pp.  74. 
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dif  f  erence 

The  remaining  portion  of  the  survey  was  dedicated  to  the  review  of 
analytical  studies  of  the  programming  techniques  used  in  mathematics 
and  science.  The  purpose  was  to  assess  their  significance  in  terms  of 
their  contributions  to  learning  theory,  value  in  choosing  from  among 
existing  program  techniques,  and  implications  for  ways  to  improve  pro¬ 
grammed  instructions.  Such  factors  as  ability,  reading  skills,  pro¬ 
gram  revision,  program  format,  response  mode,  feedback,  prompting,  item 
difficulty  and  sequence  of  items  were  contained  in  the  survey.  The 
conclusions  of  the  study  indicated  (1)  that  programmed  instruction 
possessed  a  superior  efficiency  over  regular  group  methods  of  teaching; 
(2)  that,  in  general,  students  were  favorably  disposed  to  the  new  tech¬ 
nique;  (3)  that  it  is  still  uncertain  which  group,  upper  or  lower, 
receives  the  greatest  benefits  from  programmed  instruction;  and  (4) 
that  factors  which  make  programmed  instruction  work  have  not  been 
clearly  identified.  As  in  the  previously  mentioned  studies,  the 
assessment  was  more  a  compilation  than  an  evaluation  and  involved  no 
systematically  developed  criteria. 

From  the  review  of  related  literature,  it  was  concluded  that 
most  studies  in  programmed  instruction  based  on  the  review  of  documen¬ 
tary  evidence  were  largely  compilations  with  very  little  or  no  effort 
made  to  systematically  evaluate  the  contents. 

^Leslie  J.  Briggs  and  David  Angell.  Programmed  Instruction  in 
Science  and  Mathematics.  (Washington,  D.C.:  U.S.  Office  of  Education, 
1963),  pp.  4-5.  (Mimeographed.) 
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Documentary  surveys  have  been  made  in  other  areas  of  education. 
Hall  conducted  a  bibliographical  review  of  existing  theories,  prac¬ 
tices  and  research  related  to  the  education  of  gifted  children. 

Although  the  review  was  intended  to  outline  and  evaluate  the  existing 
evidence,  very  little  evaluation  was  actually  presented  in  the  study. 
This  is  emphasized  in  his  statement  that: 

Although  some  evaluation  is  submitted,  a  final  definite  state¬ 
ment  of  the  truth  or  falsity  of  the  theories  involved  in  these 
programs  and  practices  must  await  the  verdict  of  time. 4-4- 

In  general,  Hall's  conclusions  were  based  on  opinion,  theories  and 
practices  of  others  and  were  intended  to  be  of  value  to  advanced  stu¬ 
dents,  educators  and  research  workers. 

Marrone  completed  a  study  in  which  he  collected,  analyzed  and 
evaluated  scientific  research  in  phonics  through  1957.  From  the  collec¬ 
tion,  he  compiled  accepted  generalizations  supported  by  studies  which 
were  evaluated  according  to  a  criteria  developed  by  Monroe  and  Engelhart 
The  evaluative  criteria  consisted  of  five  items  which  were  applied  to 
each  study  analyzed.  On  this  basis  the  studies  were  either  accepted  or 
rejected.  Any  one  of  the  following  reasons  were  considered  ample  justi¬ 
fication  for  rejecting  a  research  report: 

1)  groups  were  too  small;  2)  pupils  were  not  representative  of 
the  total  population;  3)  pupil  factors  and  non-exper imental  factors 

^Lynn  Gordon  Hall,  "A  Bibliographical  Survey  of  the  Education 
for  Gifted  Children"  (unpublished  Master's  dissertation,  University  of 
Alberta,  Edmonton,  1957),  p.  v. 

^Walter  S.  Monroe  and  Max  Engelhart,  A  Critical  Summary  of 
Research  Relating  to  the  Teaching  of  Arithmetic  (Urbana:  University  of 
Illinois,  University  of  Illinois  Press,  1931),  pp.  8-11. 
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were  not  controlled;  4)  statistical  procedures  were  inadequate; 

5)  methods  of  procedure  were  not  clear;  and  6)  conclusions  were 
not  substantiated  by  experimental  data. 4-6 

Even  though  the  majority  of  the  studies  were  rejected  because 
they  failed  to  comply  with  some  aspect  of  the  selected  criteria, 
Marrone  included  them  as  evidence  which  supported  the  final  conclu¬ 
sions  based  on  the  data  provided  by  the  accepted  studies.  This  tech¬ 
nique  cast  some  doubt  concerning  the  desirability  of  such  abrupt 
labeling  of  the  evaluated  research  reports.  The  fact  that  Maronne 
used  the  rejects  to  complement  the  conclusions  of  the  acceptable 
studies  suggested  a  need  for  a  procedure  which  would  have  incorporated 
a  scale  based  on  some  evaluative  gradient. 

In  concluding,  it  was  found  that  relatively  few  studies  of  the 
documentary  analysis  type  have  been  made  in  the  area  of  instruction. 

In  addition,  it  was  found  that  most  of  these  studies  seemed  to  lack 
objectivity  in  their  approach  to  evaluation  which  appeared  to  be 
largely  of  an  incidental  nature. 

X.  DESIGN  OF  THE  STUDY 

One  of  the  purposes  of  the  design  was  to  introduce  some  objec¬ 
tivity  to  the  evaluative  aspect  of  the  study.  To  achieve  this,  the 
framework  of  the  design  consisted  of  the  following  elements:  (1)  the 
questions  which  served  to  delimit  the  study;  (2)  the  development  of 

^victor  Eugene  Marrone,  "A  Critical  Analysis  of  Scientific 
Research  in  Phonics”  (unpublished  Doctoral  dissertation,  University  of 
Pittsburgh,  Pittsburgh,  1958),  pp .  17-18. 
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the  evaluative  criteria;  (3)  the  evaluation  of  the  individual  studies 
in  terms  of  the  established  criteria;  (4)  the  collection  of  data; 

(5)  the  analysis  of  the  data  in  accordance  with  the  questions  posed  in 
the  delimitation;  and  (6)  the  formation  and  assessment  of  the  con¬ 
clusions  which  served  to  answer  the  questions  comprising  the  problem. 

Questions  Constituting  the  Prob lem 

As  already  stated,  the  questions  included  were  those  which  were 
considered  to  be  most  frequently  asked  by  educators.  Specifically, 
the  development  of  the  series  of  questions  was  an  outgrowth  of  approxi¬ 
mately  two  years  of  extensive  reading  in  the  field  of  programmed 
instruction.  The  statement  of  the  questions  was  placed  in  that  portion 
of  the  study  which  was  concerned  with  the  delimitations  of  the  problem. 
The  more  specific  aspects  related  to  the  questions  were  included  in 
the  description  of  the  analysis  of  the  data. 

Development  of  the  Evaluative  Criteria 

Each  study  was  subjected  to  an  evaluative  criteria  which  was 
intended  to  determine  or  indicate  the  reliance  of  the  results.  Eight 
criteria  were  selected  for  purpose  of  evaluation.  These  were: 

1.  Duration  of  the  experiment  reported .  The  duration  of  the 
experiments  in  the  field  of  learning  was  considered  to  be  crucial  in 
contributing  to  the  accuracy  of  the  findings  by  minimizing  the  effects 
of  novelty  and  reminiscence.  Furthermore,  it  was  concluded  that 
experiments  of  a  longer  duration  represented  more  realistically  the 
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actual  or  the  practical  learning  situations  to  which  learners  are 
accustomed.  In  applying  the  criterion  of  duration,  a  period  of  three 
weeks  was  considered  to  be  of  adequate  length.  This  meant  that  the 
time  interval  between  the  initiation  and  the  termination  of  the  experi¬ 
ment  was  required  to  be  approximately  three  weeks  in  length  before  the 
criterion  of  duration  could  be  judged  as  being  satisfied. 


2.  Size  of  the  samples  used.  The  size  of  the  sample  was  an 
important  criterion  item  in  that  the  N  in  statistical  analysis  repre¬ 
sents  a  general  population.  Garrett‘S  emphasized  the  importance  of  an 
adequate  sample  size  when  he  stated  that  if: 

N  is  less  than  25,  say,  there  is  often  little  reason  for 
believing  such  small  groups  of  persons  to  be  adequately  descrip¬ 
tive  of  any  population. 

He  went  on  to  point  out  that: 

In  fact,  in  very  small  samples  widely  deviant  scores  can  hardly 
appear  in  a  random  sample  drawn  from  a  normal  group. ^ 

It  was  on  the  basis  of  Garrett's  statement  that  a  sample  of  25  was 

selected  as  the  qualifying  number  which  could  satisfy  the  criterion 

of  adequate  sample  size. 


3.  Reliability  of  the  instruments  used .  Each  study  was  examined 
for  the  presence  of  a  reliability  statement  concerning  the  measuring 
instruments.  A  report  of  the  reliability  was  considered  to  be  a  good 


^Henry  E.  Garrett,  Statistics  in  Psychology  and  Education,  (New 
York:  Longmans,  Green  and  Company,  1958),  p.  208. 
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indication  that  care  had  been  exercised  in  the  construction  or  the 
choice  of  the  instruments.  This  criterion  item  was  judged  to  be  satis¬ 
fied  if  a  report  of  the  reliability  of  the  instruments  was  included  in 
the  study. 

4.  Error  rate  of  the  program.  Since  the  programs  represented 
the  method  being  investigated  in  each  case,  it  was  considered  necessary 
to  examine  them  for  the  characteristic  which  revealed  some  reasonable 
degree  of  quality.  That  characteristic  was  concluded  to  be  the  error 
rate  because  it  indicated  in  a  very  objective  manner  that  care  had  been 
taken  by  means  of  pretesting  and  revising  in  producing  an  effective 
program.  The  criterion  item  was  considered  to  be  satisfied  if  a  state¬ 
ment  of  the  error  rate  was  included  in  the  report  of  the  study  under 
analysis . 

5.  Statistical  analysis  of  the  data .  Statistical  treatments 
such  as  the  analysis  of  variance  or  the  analysis  of  covariance  are 
modern  systematic  approaches  intended  to  provide  more  efficient  and 
exact  tests  of  experimental  hypotheses.  If  the  conclusions  of  the 
study  under  evaluation  were  based  on  some  acceptable  statistical  treat¬ 
ment  rather  than  on  the  observation  of  raw  experimental  data,  then  the 
criterion  of  the  statistical  analysis  of  data  was  considered  to  be 
satisfied. 

6.  Equating  of  the  groups.  Equating  groups  is  simply  a  method  of 
controlling  the  learner  variables  in  order  to  make  it  possible  to 
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determine  the  effectiveness  of  some  applied  treatment  or  condition  in 
the  process  of  learning.  The  criterion  of  the  equivalence  of  groups 
was  considered  satisfied  if  the  groups  participating  in  the  experiment 
were  equated  on  the  basis  of  ability  or  other  characteristics  regarded 
as  variables  to  be  controlled.  The  criterion  was  also  judged  to  be 
satisfied  if  the  statistical  treatment  of  the  data  was  of  such  a  nature 
as  to  account  for  any  observed  differences  between  the  groups. 

7 .  Pretest  relevant  to  the  sub  ject  matter  contained  in  the 
program.  It  was  decided  that  to  properly  assess  programmed  instruc¬ 
tion  or  some  variable  influencing  its  effectiveness  or  efficiency  would 
require  the  determination  of  the  achievement  level  of  the  experimental 
and  the  control  subjects.  Consequently,  to  meet  the  standard  of  the 
evaluative  criterion  required  an  administration  of  some  appropriate 
test  prior  to  the  application  of  the  experimental  factor  being  investi¬ 
gated.  In  effect,  the  pretesting  served  to  provide  the  investigator 
with  the  gain  scores  for  statistical  analysis  and  to  equate  the  subjects 
in  terms  of  their  initial  level  of  performance  within  the  particular 
content  area. 

8.  Definition  or  explanation  of  the  terms  used.  The  utility 
of  experimental  research  depends  upon  the  clarity  of  explanation. 

Since  programmed  instruction  is  a  recent  development,  the  inclusion 
of  the  meanings  of  terms  was  considered  especially  important.  The 
criterion  was  satisfied  if  a  special  section  of  the  study  was  devoted 
to  the  definition  of  terms  or  if  the  description  of  the  experiment  was 
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sufficiently  adequate  to  imply  the  meanings. 

In  the  development  of  the  evaluative  criteria,  emphasis  was 
placed  on  the  rationale  that  even  a  good  research  project  is  of  little 
value  to  a  potential  user  if  the  important  aspects  of  its  organization 
are  left  out  in  the  report. 

The  Evaluation  of  the  Individual  Studies  in  Terms  of  the  Estab lished 
Criteria 

To  facilitate  the  application  of  the  evaluative  criteria,  each 
criterion  item  was  assigned  a  value  of  one.  This  scheme  provided  for  a 
maximum  value  of  eight  for  each  study  that  satisfied  every  criterion 
item  of  the  list.  The  total  number  of  criterion  items  satisfied  by 
each  study  was  referred  to  as  its  reliance  index  and  represented  the 
relative  dependability  value  on  which  an  educator  could  base  his  deci¬ 
sions.  A  report  of  a  study,  for  instance,  with  a  reliance  index  of  6 
was  considered  to  be  more  reliable  than  one  which  possessed  an  index 
of  4.  The  reliance  index  of  each  study  was  determined  and  reported 
in  Table  I,  Pages  37  to  63,  of  the  second  chapter.  The  indices  deter¬ 
mined  through  the  application  of  the  criteria  served  as  the  basis  in 
the  assessment  of  the  conclusions  which  resulted  from  the  tabulation 
of  the  specific  data  collected  through  the  analysis  of  each  study. 

The  Collection  of  Data 

The  data  used  in  the  study  were  limited  to  experimental  research 
projects.  Only  the  findings  and  the  design  of  each  experiment  reported 
were  selected  for  the  purpose  of  development  of  the  writer's  study. 
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The  data  were  collected  by  analysis  of  theses,  dissertations,  techni¬ 
cal  reports,  reports  of  field  studies,  and  periodical  reports.  The 
Cameron  and  the  Education  Libraries  of  the  University  of  Alberta  pro¬ 
vided  the  greater  part  of  the  periodicals  in  which  a  good  number  of 
the  studies  on  programmed  instruction  were  reported.  Some  of  the 
periodicals  such  as  the  Journal  of  Programed  Instruction,  the  N.S.P.I. 
Journal,  the  AID  Magazine  and  the  Automated  Teaching  Bulletin  were 
obtained  from  the  original  sources  because  they  were  not  available  at 
the  University  Library.  A  small  number  of  theses  and  dissertations 
were  obtained  through  personal  loan  from  the  writers  and  through  the 
University  Interlibrary  Loan  Service,  A  considerable  number  of  micro¬ 
film  copies  of  the  theses  and  dissertations  were  purchased  from  the 
University  Microfilms  Inc,,  Ann  Arbor,  Michigan.  Most  of  the  technical 
reports  analyzed  were  obtained  from  the  original  sources  and  included 
such  organizations  as  The  Center  for  Programed  Instruction,  American 
Institute  for  Research,  the  U.  S,  Office  of  Education,  the  U.  S, 
Department  of  Commerce,  the  Fund  for  the  Advancement  of  Education  and 
the  Systems  Development  Corporation.  Many  reports  of  field  studies 
were  given  to  the  writer  by  individual  investigators  and  research 
departments  of  the  large  local  educational  systems.  Although  no  speci¬ 
fic  technique  was  used  in  obtaining  the  required  data,  it  may  be  con¬ 
cluded  that  correspondence  and  visitations  to  the  library  constituted 
the  data  collecting  process.  Altogether,  two  hundred  and  two  studies 
were  collected  and  analyzed  by  this  survey. 
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Analysis  of  the  Reports 

Each  report  was  analyzed  for  the  purpose  of  obtaining  the  speci¬ 
fic  information  which  was  necessary  in  forming  the  conclusions  pertinent 
to  the  particular  aspects  of  the  problem.  The  specific  data  gathered 
were:  (1)  the  subject  field  involved,  (2)  the  instructional  objectives; 

(3)  the  achievement  level  of  the  subjects;  (4)  the  ability  level  of  the 
learners,  (5)  the  special  circumstances  which  may  have  been  involved,  and 
(6)  the  reported  findings. 

Formation  and  Assessment  of  Conclusions 

The  data  from  the  analysis  were  recorded  on  five  by  eight  inch 
filing  cards.  This  method  permitted  rapid  classification  and  tabulation 
of  information  in  accordance  with  the  questions  which  delimited  the 
study.  The  tabulation  and  classification  in  most  cases  involved  the 
ability  level  of  the  subjects,  the  achievement  levels,  the  instructional 
objectives,  the  investigator's  name,  the  subject  area,  and  the  reliance 
indices.  In  this  manner  it  was  possible  to  identify  specific  conclusions, 
to  determine  the  support  they  received  from  similar  studies,  and  to 
observe  the  reliance  indices  of  the  supporting,  opposing  or  neutral 
studies . 

Each  conclusion  was  assessed  in  terms  of  the  reliance  indices  of 
the  studies  which  supported  or  did  not  support  it.  If  the  sum  of  the 
reliance  indices  supporting  the  conclusion  reached  a  total  of  eight  or 
more,  the  conclusion  was  judged  as  being  highly  dependable.  If  a  con¬ 
clusion  was  supported  by  an  indices  sum  that  was  smaller  than  eight,  it 
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was  judged  to  exist  at  a  low  dependability  level.  The  index  of  eight 
was  chosen  as  the  standard  because  it  represented  the  index  of  a  study 
which  satisfied  all  the  items  of  the  evaluative  criteria.  In  situa¬ 
tions  where  a  finding  was  supported  by  some  studies  and  opposed  by 
others,  a  dependability  level  formula  was  applied  in  which  the  sum  of 
the  supporting  indices  was  divided  by  the  sum  of  the  supporting  and 
non- support ing  indices.  The  ratio  resulting  from  the  application  of 
this  formula  served  as  an  objective  and  expedient  expression  of  the 
degree  of  support  that  any  particular  finding  received  from  the  reliance 
indices  representing  the  total  available  research  dealing  with  that 
specific  finding.  A  minimum  ratio  of  0.9  was  required  before  a  con¬ 
clusion  was  assessed  as  highly  dependable.  Because  a  highly  reliable 
study  was  one  which  satisfied  all  eight  of  the  items  of  the  evaluative 
criteria,  it  was  necessary  to  set  the  ratio  at  0.9  in  order  to  prevent 
inconsistencies  which  would  have  produced  by  high  dependability  values 
resulting  from  small  supporting  and  non- supporting  indices.  Briefly, 
the  minimum  ratio  of  0.9  prevented  a  situation  in  which  a  low  supporting 
reliance  index  could  result  in  a  highly  dependable  finding.  Had  a  mini¬ 
mum  ratio  of  0.7  been  used  in  lieu  of  0.9,  it  could  have  been  possible 
to  achieve  a  high  dependability  level  with  a  supporting  reliance  index 
of  three  and  a  non- supporting  index  of  one.  In  this  study,  a  reliance 
index  of  three  was  not  considered  an  indication  of  high  dependability. 
Briefly,  the  minimum  ratio  served  in  demanding  a  high  percentage  of 
support  and  also  required  reference  to  a  greater  number  of  studies  in 
presence  of  contradictory  evidence  before  a  conclusion  could  be  rated 
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as  highly  dependable. 

In  summary,  the  design  comprised  the  evaluation  of  research 
reports  and  the  application  of  the  evaluative  data  to  the  assessment 
of  the  observed  conclusions. 


CHAPTER  II 


EVALUATION  OF  THE  REPORTS  AND  FINDINGS 

The  first  portion  of  this  chapter  included  the  evaluation  of 
the  documents  from  which  the  data  were  obtained.  The  second  part  was 
concerned  with  the  assessment  of  the  conclusions  revealed  by  the 
tabulated  data. 

I.  EVALUATION  OF  THE  STUDIES 

The  studies  from  which  the  required  information  was  obtained 
were  evaluated  according  to  an  established  evaluating  criteria. 

Because  it  was  the  purpose  of  this  study  to  present  a  detailed  evalu¬ 
ation  as  well  as  a  general  picture,  the  results  of  the  first  evaluation 
were  expressed  in  terms  of  the  individual  index  numbers  which  repre¬ 
sented  the  number  of  criterion  items  that  were  satisfied  by  each  study, 
and  in  terms  of  the  total  of  all  the  indices  for  each  criterion  item. 
The  details  of  the  individual  evaluations  were  included  in  Table  I, 
page  37.  To  facilitate  tabulation,  the  criterion  items  were  referred 
to  by  the  following  letters:  (A)  definition  or  explanation  of  terms, 
(B)  size  of  sample,  (C)  duration  of  the  experiment,  (D)  pretest  rele¬ 
vant  to  the  subject  matter  contained  in  the  program,  (E)  reliability 
of  the  instruments,  (F)  error  rate  of  the  program,  (G)  equating  of 
groups,  and  (H)  statistical  analysis  of  data. 
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Observation  of  Table  II,  page  65,  revealed  a  number  of  charac¬ 
teristics  of  the  surveyed  literature.  Only  14  percent  of  the  available 
literature  reported  error  rates  of  the  programs  which  were  used  in  the 
experiments.  Since  this  finding  suggested  that  the  remaining  studies 
used  programs  which  may  or  may  not  have  been  carefully  constructed, 
there  was  no  basis  on  which  to  assume  that  research  in  general  utilized 
pretested  programs  in  determining  the  effectiveness  of  the  various 
aspects  of  programmed  instruction.  Definitions  or  explanations  of 
terms,  factors  crucial  to  the  proper  interpretation  of  the  findings, 
appeared  in  only  18  percent  of  the  studies  analyzed.  It  seemed  in  most 
cases  that  the  literature  assumed  only  one  meaning  per  term  and  that 
the  reader  was  aware  of  this  assumption.  Statements  of  the  reliability 
of  the  instruments  used  were  made  in  only  26  percent  of  the  studies.  It 
was  apparent  that  in  many  instances  the  investigators  assumed  that  the 
reliability  was  adequate  because  standardized  instruments  were  used  in 
measuring  the  effects  of  the  variables  being  investigated.  Only  45  per¬ 
cent  of  the  studies  involved  durations  of  time  equal  to  or  exceeding 
the  minimum  of  three  weeks  as  required  by  the  criteria.  An  adequate 
sample  of  25  or  more  was  achieved  in  54  percent  of  the  experiments, 
that  is,  only  slightly  more  than  half  of  the  groups  used  were  consi¬ 
dered  to  be  adequately  representative  of  the  population  from  which  they 
were  presumably  chosen.  Pretesting  to  determine  the  initial  performance 
level  of  the  subjects  was  done  in  63  percent  of  the  studies.  Other 
investigations  assumed  that  the  initial  performance  level  of  the  groups 
was  either  equal  for  all  groups  or  not  seriously  different  because  of 
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TABLE  II 

TOTAL  AND  PERCENTAGE  OF  STUDIES  SATISFYING 
EACH  ITEM  OF  THE  EVALUATIVE  CRITERIA 


Criterion  Item 

Totala 

Percentages^ 

Definition  or  explanation  of  terms 

35 

18% 

Size  of  the  sample  used 

108 

54% 

Duration  of  the  experiment 

89 

45% 

Pretesting  to  determine  initial 
performance  level 

125 

63% 

Reliability  of  the  measuring  instruments 

51 

26% 

Error  rate  of  the  program 

27 

14% 

Equating  of  groups 

138 

69% 

Statistical  analysis  of  data 

172 

86% 

a  Total  of  studies  satisfying  each  item  of  the  evaluative 
criteria. 

D  Percentage  of  studies  satisfying  each  item  of  the  evaluative 


criteria. 
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the  type  of  statistical  treatment  used.  In  31  percent  of  the  studies 
analyzed  no  attempt  had  been  made  to  account  for  any  differences  between 
the  groups.  In  most  cases  the  groups  were  assumed  to  be  sufficiently 
equivalent  with  respect  to  the  various  learner  variables.  The  most 
adequate  performance  in  reporting  involved  the  statistical  analysis  of 
data.  Eighty  -  six  percent  of  the  reports  indicated  that  some  accept¬ 
able  approach  had  been  used  in  the  interpretation  of  the  data.  In  sum¬ 
marizing,  it  was  concluded  that  the  evaluation  of  the  reports  revealed 
major  weaknesses  with  respect  to  the  explanation  of  the  terms,  duration 
of  the  experiments,  reliability  of  the  instruments  used  and  the  descrip¬ 
tion  of  the  programs  in  terms  of  the  error  rate.  The  two  somewhat  less 
serious  weaknesses  were  found  to  be  the  size  of  the  samples  used  and 
the  lack  of  pretesting  to  determine  the  initial  level  of  performance. 

II.  THE  FINDINGS  AND  THEIR  EVALUATION 

To  identify  and  evaluate  the  specific  findings  related  to  the 
questions  posed  in  the  delimitation,  tables  were  so  constructed  as  to 
include  the  supporting,  the  non-supporting  and  the  opposing  indices 
along  with  the  data  gathered  during  the  analysis.  In  this  manner  it 
was  possible  to  reveal  the  specific  findings  and  at  the  same  time  to 
assess  their  dependability  by  applying  the  dependability  formula  to 
reliance  indices  of  the  supporting  literature. 

Programmed  Ins  true tion  and  the  Exceptional  Cases 

After  examining  the  literature  gathered  for  this  report,  it  was 


67 


found  that  very  few  studies  were  concerned  with  the  comparison  of  pro¬ 
grammed  instruction  and  the  conventional  techniques  in  situations 
differing  from  what  would  be  expected  within  a  normal  classroom.  A 
portion  of  the  studies  dealing  with  special  cases  involved  comparisons 
of  the  effectiveness  of  program  characteristics.  These  reports  were 
included  in  the  more  appropriate  sections  of  this  study. 

Observation  of  Table  III  indicated  that  the  data  gathered  was 
insufficient  to  reveal  more  than  one  specific  finding  which  possessed  a 
high  level  of  dependability.  This  finding  showed  that  teaching  the 
recognition  of  words  in  spelling  or  reading  to  retarded  pupils  by  means 
of  programmed  instruction  was  at  least  as  effective  as  teaching  the  same 
material  through  the  use  of  the  more  conventional  techniques.  From 
the  more  general  point  of  view  it  appeared  that  programmed  instruction 
was  at  least  as  effective  as  conventional  procedures  with  some  tendency 
toward  being  superior.  This  indication  was  supported  by  the  fact  that 
no  study  was  reported  in  which  the  conventional  methods  were  found  to 
be  more  effective. 

Programmed  Ins  truction  Within  the  Various  Sub  ject  F ie Ids 

Although  a  considerable  number  of  investigations  were  concerned 
with  the  effectiveness  and  efficiency  of  programmed  instruction  as  com¬ 
pared  to  the  conventional  methods,  this  abundance  of  research  has  been 
restricted  to  a  surprizingly  limited  number  of  subject  areas.  The 
greatest  portion  of  the  studies  were  conducted  within  the  fields  of 
mathematics,  physical  science,  English,  and  psychology. 
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Programmed  instruction  in  mathematics .  Specific  conclusions  in 
this  section  were  derived  from  a  set  of  figures  based  on  Table  IV,  page  71. 
Each  figure  was  designed  to  represent  the  findings  in  terms  of: 

(1)  the  specific  content  area,  (2)  the  achievement  level  of  the 
experimental  and  control  groups,  (3)  the  ability  level  of  the  learners, 

(4)  the  instructional  objectives  and  (5)  the  reliance  indices  of  the 
supporting,  opposing  or  the  neutral  studies.  Only  those  findings  which 
rated  high  in  dependability  were  referred  to  in  this  portion  of  the 
study. 

Although  there  was  a  tendency  to  favor  programmed  instruction 
in  mathematics  at  the  elementary  level,  only  one  finding  was  rated  as 
highly  dependable.  It  revealed  that  programmed  instruction  was  equally 
effective  to  conventional  methods  in  teaching  problem  solving  in 
mathematics  to  average  ability  groups  at  the  elementary  school  level. 
Figure  1,  page  76,  represents  the  summary  and  evaluation  of  the 
f indings . 

An  even  greater  tendency  was  found  to  favor  programmed  instruc¬ 
tion  at  the  junior  high  school  level.  In  fact,  only  one  of  the  avail¬ 
able  studies  supported  conventional  instruction  as  being  superior  to 
programmed  instruction.  However,  in  spite  of  all  the  evidence 
supporting  programmed  instruction,  only  one  specific  finding  was  rated 
as  being  highly  dependable.  It  indicated  that  programmed  instruction 
was  equally  effective  to  conventional  instruction  in  teaching  or  in 
attaining  unspecified  learning  objectives  at  the  junior  high  school 
level  involving  heterogeneous  ability  groups.  Figure  2,  page  77, 
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summarizes  the  findings  and  their  evaluation. 

Out  of  the  eight  specific  findings  indicated  in  Figure  3,  page  79, 
not  one  was  rated  as  highly  dependable,  although  the  general  trend 
seemed  to  indicate  that  programmed  instruction  and  conventional  tech¬ 
niques  were  equally  effective  in  teaching  mathematics  at  the  senior 
high  school  level.  Figure  3  also  suggested  that  actually  very  little 
has  been  done  with  programmed  instruction  in  mathematics  at  the  high 
school  level. 

Figure  4,  page  80,  implied  that  perhaps  programmed  instruction 
in  mathematics  could  be  most  successful  in  comparison  to  the  conven¬ 
tional  methods  at  the  college  level.  Except  for  one  specific  case, 
all  the  findings  implying  that  programmed  instruction  was  at  least  as 
effective  as  the  traditional  methods  were  rated  as  highly  dependable. 
Briefly,  the  findings  indicated  that  programmed  instruction  was 
equally  effective  to  conventional  techniques  in  teaching  concepts, 
understandings,  application  of  things  learned,  and  transfer  in  mathe¬ 
matics  at  the  college  level.  Since  no  attempt  was  made  in  any  of  the 
studies  to  deal  with  different  ability  levels,  it  was  assumed  that  the 
college  level  was  synonymous  with  at  least  the  average  ability  level. 

In  numerous  cases  the  instructional  objectives  were  not  specified. 

These  studies,  however,  supported  the  specific  findings  at  the  high 
level  of  dependability. 

Programmed  instruction  in  English.  Table  V,  page  81,  includes 
the  available  studies  which  compared  programmed  instruction  to 


Objectives  High  Ability  Average  Ability  Low  Ability  Mixed  Abilities 


79 


u 


p-i 


cj 


PM 


cj 


w 

z 


PM 


o 

HM 

O 


M 

z 


cd 

HM 

Pm 


CO 

m 

in 

m 

in 

m 

CO 

W) 

X) 

d 

CD 

d 

•H 

cn 

X) 

o 

MM 

cd 

d 

•r4 

•r-l 

cn 

1 — 1 

cd 

4J 

Li 

O 

4-J 

PL 

■M 

Cd 

a) 

a) 

PL 

•H 

CO 

CJ 

MM 

CL 

cn 

CD 

O 

d 

•H 

CO 

CO 

4-1 

O 

d 

0) 

H 

d 

o 

Cl 

■rM 

XI 

PL 

cd 

4-) 

cd 

O 

d 

d 

a 

d 

o 

Pm 

o 

PM 

£3 

<3 

H 

2 

h-J 

<3 

2 

O 

HM 

H 

2 

PJ 

r-J 

> 

PJ 

2 

> 

O 

PJ 

CJ 

r-J 

P 

hJ 

JZ 

o 

<2 

o 

£d 

2 

CJ 

o 

CO 

l-M 

H 

PC 

CJ 

CJ 

£3 

l-M 

Pd 

PC 

H 

CO 

Pd 

2 

o 

M 

HM 

d 

2 

o 

P 

PJ 

•rM 

co 

4-J 

vj 

o 

CO 

Jr 

PJ 

PJ 

<q 

PC 

d 

ex 

H 

4-1 

(LI 

Q 

CO 

s 

O 

H 

d 

£3 

Qd 

<3 

•rM 

O 

CL 

M 

CO 

X) 

Pm 

Pm 

CJ 

1) 

O 

1 — 1 

£ 

H 

g 

CO 

<t] 

cd 

CO 

5m 

PJ 

PJ 

P0 

2 

PC 

o 

PJ 

H 

5m 

> 

<c 

PL 

l-M 

y 

H 

PC 

CJ 

2 

d 

CO 

PJ 

HM 

•H 

CD 

Pm 

5m 

O 

Pm 

CO 

o 

•rM 

PJ 

P 

> 

X) 

O 

cd 

d 

PJ 

PC 

4M 

•rM 

> 

H 

w 

PJ 

CO 

CD 

H 

§ 

0) 

O 

<3 

O 

d 

Pd 

•H 

cd 

<3 

X) 

•rM 

PM 

d 

t-M 

2 

•H 

CD 

o 

d 

CJ 

a) 

o 

r-M 

d 

cd 

cd 

d 

•H 

4-J 

r-H 

P) 

<u 

CD 

pd 

2 

cd  X>  O 


Reliance  indices  favoring  conventional  instruction 


Objectives  High  Ability  Average  Ability  Low  Ability  Mixed  Abilities 


80 


o 


w 

P 


p 


M 

P 


CJ 


M 

P 


P 

M 

p 


Cd 

M 

P-i 


<r 

p 

rH 

cn 

00 

00 

rH 
r— 1 

m 

T— 1 

CO 

CO 

CU 

r— 1 

CO 

00 

c 

•H 

P 

0 

cd 

0 

O 

•H 

4-1 

o 

P 

<U 

•r4 

<44 

4-1 

04 

4-1 

cd 

cu 

•r4 

04 

•H 

CO 

o 

144 

o 

CO 

0) 

O 

o 

•r-l 

CO 

<u 

44 

o 

0 

<U 

rH 

0 

O- 

o 

0 

•H 

P 

P 

cd 

CO 

cd 

O 

0 

C 

Cu 

o 

C 

P 

O 

P 

P 

c 

H 

P 

P 

g 

M 

H 

2 

W 

> 

§ 

o 


Q 

P 

P 

w 

<d 

> 

w 

p 

p 

o 

M 

w 

H 

o 

U 

w 

P 

p 

P 

p 

H 

o 

CO 

o 

p 

M 

w 

p 

o 

H 

w 

p 

H 

P 

p 

C 

p 

2 

o 

p 

CO 

i — i 

p 

8 

O 

M 

p 

H 

p 

<j 

p 

W 

o 

P 

H 

CO 

CO 

p 

W 

P 

P 

!> 

M 

>> 

M 

co 

H 

P 

C_) 

o 

w 

p 

p 

w 


H 

S 

C 

p 

§ 

o 


e 


o 

•r4 

44 

o 

0 

S4 

44 

CO 

0 

•r4 

P 

CU 

cd 

o 

00 

o 

o 

o 

00 

c 

CO 

•r4 

CU 

o 

o 

o 

•r4 

> 

P 

cd 

0 

<44 

•H 

CO 

CU 

cu 

o 

o 

0 

•r4 

cd 

P 

•H 

0 

rH 

•H 

CU 

o 

cu 

o 

r-4 

0 

cd 

cd 

>4 

•H 

44 

rH 

0 

CU 

cu 

P 

p 

cd  P  CJ 


Reliance  indices  favoring  conventional  instruction 


81 


a 

w 

u 


rD 
I - 1 

s 


m 


m 


in 


w 

fa 

PQ 

s 


fa 

<3 

2 

O 

w 

H 

!Z 

W 

> 

3Z 

O 

(fa 


CO 

co 
W 
53 

g 

M 

H 
<fa 

w 

fa 
fa 
fa 

fa 

g 

H  H 

o  y 


:s 

o 


s 


Pd 

co 

M 

fa 

(fa 

33 

fa 


co 

Q 

O 

Pd 

H 


O 

fa 

Pd 

fa 


w 

CO 

3 


cd 

I — I 
fa 


in 


ccS 


cd 

CO 

fa 

cd 

fa 

cd 

fa 

o 

CU 

a) 

o 

CU 

O 

CU 

•H 

> 

•H 

•H 

•H 

•H 

•H 

4-1 

•H 

CM 

CO  4-1 

CM 

co  m 

IM 

O 

4-1 

•H 

4J  cd 

•H 

M  cd 

•H 

P> 

O 

O 

fa  o 

O 

fa  o 

O 

Ci 

o 

CU 

(U  *H 

CU 

<U  -H 

<U 

4-1 

a, 

O  .-H 

fa 

O  i— l 

fa 

CO 

fa 

CO 

cd  fa 

co 

cd  fa 

CO 

cd 

o 

id 

O  fa 

Cd 

O  fa 

d 

M 

fa 

fa  <3 

fa 

fa  c 

fa 

a) 

> 

0) 


Pn 

CU 

cu 

M 

toO 

toO 

•H 

cd 

cd 

< — 1 

fa 

M 

M 

•H 

toO 

CU 

£ 

CU 

fa 

•H 

> 

o 

P> 

<3 

Pd 

C 

fa 

<3 

fa 

fa 

<U 

CU 

M 

•H 

cd 

CM 

CU 

u 

CU 

•rH 

toO 

CU 

toO 

o 

35 

cd 

r— 1 

cd 

CU 

O 

M 

CU 

u  $ 

fa 

rH 

CU 

o 

CU  O 

CO 

CU 

> 

o 

>  <— 1 

cd 

PQ 

cd 

<3 

<3  co 

fa 

M 

r-H 

rH 

rH 

cd 

o 

O 

O 

Pn 

CU 

o 

O 

o 

M 

E  *  * 

fa 

fa 

fa 

o 

cd 

a>  cu 

o 

o> 

a 

o 

r-H 

+J  <t 

cu 

>  > 

M 

CO 

M 

CO 

M 

CO 

cd 

toO 

CU  CU 

O 

(U 

O 

cu 

O 

cu 

cu  <u 

cu 

•H  i — 1 

•H 

fa 

fa 

•rH 

fa 

fa 

•H 

fa 

fa 

E  fa 

r-H 

fa 

cd 

toO 

cd 

cd 

too 

cd 

cd 

toO 

cd 

a)  cd 

r-H 

a 

p 

•H 

M 

P 

•H 

M 

CU 

•H 

M 

•M  M 

o 

<3 

fa 

Pd 

o 

fa 

Pd 

(fa 

CO 

Pd 

(fa 

w  (fa 

c_> 

cd 

o 

-H 


Cd 

bO 

•H 

m 

CO 

(U 

> 

id 


o 

td 

toO 

•H 

Cd 

fa  o 

p^ 

co  B 

•H 

toO  M 

fa 

O  O 

ci 

PJ  -M 

CO 

M  CO 

fa  cd 

O  Cu 

rH 

fa  -M 

O 

N  cd 

M  cd 

•H 

O  (U 

M 

P  cd 

O  fa 

PQ 

PQ  (fa 

PQ 

PQ  2 

Q  CO 

id 


O 

•H 

M 

O 

P> 

M 

M 

CO 

cd 

•H 

fa 

CU 

i 

r-H 

M 

cd 

toO 

M 

o 

M 

M 

PJ 

CU 

0) 

cd 

bO 

cd 

cu 

•H 

M 

M 

cd 

M 

o 

fa 

CU 

o 

CU 

•H 

pl 

fa 

CO 

2 

CO 

CO 

CU 

CU 

o 

o 

•H 

•H 

fa 

fa 

cd 

cd 

•H 

•H 

CU 

CU 

O 

o 

cd 

cd 

cd 

cd 

•H 

•H 

rH 

i — 1 

cu 

CU 

fa 

fa 

cd  x>  o 


Reliance  indices  supporting  conventional  instruction 


82 


u 


<$  LO  <t 


m 


03 


d 

CO 

d 

•H 

X 

d 

X 

d 

/"N 

o 

0) 

CO 

o 

CO 

a) 

o 

cu 

o 

XI 

•H 

> 

cu 

•H 

d 

•H 

•d 

•H 

•H 

0) 

4-J 

•d 

rH 

CO 

4-J 

a; 

4-4 

CO 

4-1 

M-l 

d 

4-1 

d 

pj 

o 

4-J 

a 

4-1 

03 

jd 

•H 

4J 

03 

•r-l 

(D 

•H 

CU 

d 

d 

u 

•H 

a 

a 

cu 

o 

C / ) 

PL 

O 

O 

M-4 

CO 

M-4 

•H 

d 

<U 

o 

ai 

•d 

CO 

CO 

d 

cu 

rH 

CU 

•d 

CU 

CO 

CO 

•H 

CO 

4-1 

4-J 

d 

o 

i — 1 

4-1 

4-1 

a 

a 

rH 

o 

i—4 

a, 

d 

4-1 

d 

d 

d 

CO 

JO 

•H 

d 

PL 

a 

o 

g 

CO 

•H 

d 

PL 

CO 

03 

O 

cr 

03 

o 

d 

o 

d 

o 

PL 

aj 

03 

o 

d 

-x 

o 

PL 

d 

d 

03 

o 

d 

o 

M 

Ph 

u 

c 

Pm 

fd 

CJ 

po 

cn 

o 

< 

H 

(d 

<d 

H 

W 

CU 

P 

> 

X 

PO 

<u 

cu 

C 

rH 

•H 

CO 

H 

M-l 

CU 

cu 

•d 

a> 

CD 

•H 

<u 

4-1 

00 

CD 

00 

00 

-U 

00 

•H 

o3 

CD 

o3 

03 

X 

•H 

03 

rH 

d 

PL 

d 

Jd 

jd 

d 

cu 

rH 

jd 

d 

•H 

CU 

CO 

2 

CU 

00 

00 

CU 

X 

•H 

00 

CU 

X 1 

> 

d 

o 

> 

•d 

•H 

> 

*H 

x 

•d 

> 

c 

C 

d> 

di 

C 

Pd 

Pd 

C 

s 

Pd 

< 

d 

O 


£ 

o 


4-1 

d 

>-> 

cu 

d 

d 

B 

03 

03 

CU 

rH 

4-> 

X 

4-J 

> 

cu 

d 

d 

cu 

> 

CU 

cu 

CU 

•rH 

cu 

6 

X 

B 

jd 

t— — 1 

<U 

o3 

cu 

CD 

T— 1 

d 

r-H 

< 

W 

O 

w 

>•> 

>> 

d 

d 

d 

03 

03 

o3 

4-1 

4-1 

4-1 

d 

d 

CO 

d 

CU 

CU 

cu 

CD 

m 

(U 

cu 

g 

X 

g 

X 

g 

X 

CU 

Oj 

<u 

03 

X 

CU 

03 

r-H 

d 

r-H 

d 

d 

r—l 

d 

w 

O 

w 

O 

o3 

w 

O 

r> 


r—l 

00 

r-H 

o 

o 

o 

r\ 

o 

o 

X 

r- 

X 

r— 1 

a 

CD 

d 

co 

CO 

d 

CO 

CO 

r—l 

O 

cu 

O' 

O 

CU 

r-H 

•d 

jd 

X 

•rH 

Jd 

X 

d 

00 

o3 

X 

d 

00 

05 

X 

d 

•d 

d 

d 

QJ 

•rH 

d 

d 

X 

Pd 

O 

03 

co 

Pd 

O 

o3 

d 

o 

•H 

4-1 

oj 


00 

d 

•d 

o 

d 

4-1 

-P  rH 

d 

d 

cu 

CO 

d  JO 

o 

ai 

g 

cu 

cu  g 

CO 

jo 

r-H 

> 

00  o 

i — i 

d 

rH 

d 

X 

i — i 

o3 

•H 

M 

W  Ed 

W 

PO 

d 

CO 

d 

•d  03 

rH 

JO 

CU 

D*  CD 

rH 

O 

H 

4-1  CO 

cu 

CD 

rH 

O  oj 

d 

oj 

•d 

o  s 

O 

X 

S 

O'Hare  Elementary  Average  Comprehension 

Grades  3  Transfer 

and  4  Memorization 


TABLE  V  (continued) 


83 


u 


fd 


rd 

50 

CO 

50 

d 

d 

d 

d 

CO 

•H 

X 

X 

X) 

X) 

X 

X 

X 

•d 

X 

o 

CU 

u 

X 

<U 

CU 

a) 

CU 

CU 

a) 

CU 

X 

CU 

•H 

> 

o 

X 

d 

•H 

•H 

•d 

•H 

•H 

•d 

•d 

d 

•d 

■u 

•H 

5-1 

d 

CU 

d 

d 

Id 

Ld 

Id 

4-4 

d 

Ld 

Id 

Ld 

CO 

d 

d 

Ld 

o 

4J 

CU 

•H 

■u 

CU 

•H 

•H 

•d 

•H 

CU 

•d 

•d 

•d 

4J 

4J 

CU 

•d 

d 

O 

IM 

CO 

4-4 

CO 

4-1 

o 

a 

o 

o 

Ld 

o 

CJ 

o 

PX 

CO 

id 

o 

5-i 

<U 

CO 

5-i 

•d 

5-i 

CO 

CU 

CU 

CU 

<u 

CO 

CU 

CU 

CU 

CO 

CU 

d 

CO 

<U 

4J 

d 

0) 

O 

CU 

d 

a 

cx 

PX 

PI¬ 

d 

Pi¬ 

PI¬ 

Pi¬ 

4-1 

O 

(U 

d 

Pi¬ 

CO 

P 

d 

-d 

CU 

X 

d 

CO 

CO 

CO 

CO 

d 

co 

CO 

co 

O 

d 

X 

d 

co 

d 

o 

5-1 

4J 

a 

d 

d 

d 

d 

d 

d 

d 

d 

d 

d 

d 

o 

d 

d 

d 

H 

H 

O 

CO 

P 

H 

P 

P 

P 

P 

H 

P 

P 

P 

Pm 

o 

P 

H 

P 

d 

o 


d 

50 

•H 

4J 

co 

CU 

> 

c 


X 


LO 


vfr 


LO 


lO 


CO 


m 


m 


lO 


rH 

CU 

CU 

50 

> 

X 

X 

X 

d 

CU 

CU 

CU 

CU 

d 

rH 

CO 

•d 

•d 

CO 

•d 

CU 

<U 

Ld 

Ld 

CU 

Ld 

> 

Po 

•d 

•d 

CU 

CU 

•d 

•d 

•d 

(U 

CU  < 

4-1 

4-1 

o 

50 

50 

o 

4-1 

O 

50 

50 

•H 

X 

•d 

CU 

d 

d 

(U 

X 

*d 

CU 

d 

d  £ 

r- H 

CU 

\ — 1 

Pi¬ 

P 

d 

d 

P 

Pi¬ 

CU 

pd 

Pi¬ 

P 

d 

d  O 

•rH 

x 

•d 

co 

50 

CU 

15 

£ 

CU 

50 

co 

X 

•d 

co 

50 

CU 

£ 

<u  r— 1 

p 

•pH 

P 

d 

•d 

> 

O 

o 

> 

•d 

d 

•d 

P 

d 

•H 

> 

o 

>  CU 

S 

d 

P 

P 

< 

P 

p 

<J 

EC 

P 

s 

d 

p 

Ed 

< 

p 

<  pci 

4J 

i — 1 

rH 

rH 

rH 

rH 

rH 

d 

En 

o 

o 

o 

o 

o 

o 

CU 

d 

o 

o 

o 

o 

o 

o 

B 

d 

-d 

o 

P 

rd 

rd 

P 

P 

o 

a) 

r— 1 

4-1 

P 

cu 

o 

r—H 

o 

oo 

CU 

o 

C7N 

CJ 

CTN 

o 

ON 

CJ 

rH 

> 

CU 

d 

50 

d 

CO 

d 

CO 

50 

d 

co 

d 

co 

d 

co 

d 

co 

CU 

> 

<u 

CU 

CU 

O 

<u 

O 

cu 

CU 

O 

CU 

O 

cu 

o 

CU 

O 

cu 

•rH 

CU 

B 

X 

rH 

•rH 

-d 

X 

•rH 

P 

X 

rH 

•d 

rd 

X 

•d 

rd 

X 

•H 

P 

X 

•H 

P 

X 

p 

r— 1 

cu 

d 

i — 1 

d 

50 

d 

d 

50 

d 

i — 1 

d 

50 

d 

d 

50 

d 

d 

50 

d 

d 

50 

d 

CJ 

r—H 

d 

o 

CU 

•d 

d 

d 

•d 

d 

o 

d 

•d 

d 

d 

•d 

d 

d 

•d 

d 

<u 

•d 

d 

< 

w 

O 

o 

c o 

p 

O 

X 

p 

O 

o 

X 

P 

e> 

X 

EC 

o 

X 

P 

O 

CO 

EC 

O 

5-1 

CU 

4- 1 

5- i 
O 
Pm 


d 

cu 

cci 


d 

d 

X  B 

<U 

<U  d 

pc!  P 


P 

cu 

CJ 

B 

d 

d 

rX 

d 

4J 

CU 

o 

o 

d 

d 

d 

CJ 

4-1 

cu 

•d 

CU 

cu 

d 

d 

d 

d 

X 

rH 

CO 

PX 

cu 

d 

o 

X 

Pi 

CU 

cu 

a 

4J 

4-1 

d 

d 

DPI 

X 

co 

CO 

CO 

84 


M 

a 

IN 

m 

i-"* 

M 

PM 

CO 

H 

bO 

bO 

d 

d 

d  CO 

•H 

d 

•rH 

P 

/  \ 

o  <u 

P 

o 

P 

CU 

X 

•H  > 

d 

•rH 

d 

*p4 

CU 

4-J  *H 

CO 

d 

4-J 

d 

<4M 

d 

O  -U 

4-J 

4-J 

d 

4-J 

•H 

a 

d  a 

a 

CO 

o 

CO 

O 

•H 

5m  cu 

CO 

CU 

5m 

•H 

54 

CU 

4-J 

4-J  *T— ) 

4-1 

o 

CU 

r-M 

CU 

a 

d 

CO  -Q 

o 

d 

X 

CM 

X 

CO 

o 

d  o 

cd 

o 

d 

a 

d 

d 

o 

M 

Pm 

o 

P 

< 

P 

P 

> 

w 

r— 1 

P 

a) 

PQ 

> 

<3 

a) 

H 

t— H 

CO 

CO 

CU 

CU 

>% 

•H 

•H 

4-1 

4-J 

4-J 

•H 

TO 

•r4 

p 

*r4 

< — 1 

<U 

r-M 

cu 

rM 

•rl 

X 

*r4 

X 

•r4 

2 

X 

•rl 

X 

•H 

P 

o 

< 

a 

d 

a 

d 

p 

4-> 

r—M 

r-M 

d 

o 

o 

cu 

o 

o 

e 

-d 

X 

d)  i — l 

o 

ON 

o 

ON 

>  cu 

5m 

CO 

5m 

CO 

5m 

<u  > 

O 

cu 

o 

CU 

O 

•H  cu 

•H 

-d 

P 

•rH 

X 

P 

•H 

X 

d 

bO 

d 

d 

b 0 

d 

d 

o 

d 

•rH 

5m 

d 

•H 

5m 

d 

<3 

P 

X 

O 

p 

Pd 

O 

p 

d 

o 

•H 

4J 

cti 

bO 

r-H 

•H 

a 

r-H 

4-1 

•rH 

•H 

CO 

5m 

d 

CU 

TO 

CO 

-X 

> 

r-H 

d 

X 

d 

ctf 

•H 

d 

M 

& 

12 

M 

CU 

bO 

ctJ 

u  x 

a)  bO 

f>  *H 


O 

O 

X 

o 

co  co 
cu 

-d  T3 
bO  d 

•H  5-i 

P  O 


oo 

P 

d 

d 


85 


conventional  instruction  in  English  at  the  elementary,  junior  high 
school,  senior  high  school  and  the  college  levels.  Figures  based  on 
the  data  found  in  this  table  were  constructed  for  the  purpose  of 
revealing  the  findings  and  their  level  of  dependability  in  a  specific 
manner . 

At  the  elementary  level  four  high  level  conclusions  were 
identified  by  Figure  5,  page  86.  These  were:  programmed  instruction 
and  conventional  methods  are  equally  effective  in  teaching  concepts 
to  high  ability  groups,  programmed  instruction  and  conventional  instruc 
tion  are  equally  effective  in  teaching  facts  to  average  groups,  the 
two  methods  are  equally  effective  in  teaching  concepts  to  average 
ability  groups  and  programmed  instruction  and  conventional  methods 
are  equally  effective  in  teaching  unspecified  learning  material  to 
average  ability  groups.  No  study  was  found  to  support  conventional 
instruction  as  a  superior  method  to  programmed  techniques. 

At  the  junior  high  school  level  Figure  6,  page  87,  showed  that: 
programmed  instruction  and  conventional  instruction  are  equally  effec¬ 
tive  in  teaching  understanding  to  mixed  ability  groups  and  that  the 
two  methods  are  also  equally  effective  in  achieving  unspecified 
learning  objectives  with  the  mixed  ability  groups.  A  general  tendency 
to  favor  conventional  instruction  at  the  three  ability  levels  was  noted 
however,  none  of  the  specific  findings  supporting  traditional  methods 
existed  at  the  high  dependability  level. 

At  the  senior  high  school  level  three  highly  dependable  findings 
were  identified  by  Figure  7,  page  88.  Programmed  instruction  was  found 
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to  be  equally  effective  to  conventional  instruction  in  achieving  unspeci¬ 
fied  learning  objectives  at  the  average  ability  level.  Programmed 
instruction  was  found  to  be  more  effective  than  conventional  methods 
in  achieving  unspecified  learning  objectives  at  the  high  ability  level. 
The  two  methods  were  found  to  be  equally  effective  in  producing  learning 
at  the  low  ability  level.  It  is  doubtful  if  any  findings  would  have  been 
rated  as  highly  dependable  if  the  instructional  objectives  had  been 
specified.  The  studies  generally  tended  to  support  programmed  instruc¬ 
tion  as  the  more  effective  method  of  instruction. 

Figure  8,  page  90,  revealed  one  high  level  finding  at  the  college 
level.  It  indicated  that  programmed  instruction  and  the  conventional 
methods  were  equally  effective  in  achieving  unspecified  learning  objec¬ 
tives  at  the  mixed  ability  level.  Here,  again,  it  is  very  unlikely 
that  any  finding  would  have  been  rated  highly  had  the  abilities  and 
the  instructional  purposes  been  specified.  No  evidence  was  found  to 
support  conventional  instruction  as  a  better  method. 

Programmed  instruction  in  science .  Examination  of  Table  VI, 
page  91,  did  not  reveal  any  findings  which  existed  at  a  high  level  of 
dependability.  Here,  as  in  the  field  of  mathematics,  a  tendency  to 
favor  programmed  instruction  was  noted  even  though  none  of  the  specific 
findings  could  be  highly  rated.  No  available  literature  was  found, 
however,  to  support  conventional  instruction  as  a  more  effective  means 


of  producing  learning. 


Objectives  High  Ability  Average  Ability  Low  Ability  Mixed  Abilities 
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Programmed  ins truction  in  psychology .  Only  three  available 
reports  compared  programmed  instruction  with  the  conventional  methods 
in  teaching  psychology.  All  three  experiments  were  conducted  at  the 
college  level.  Though  none  of  the  findings  could  be  rated  high  in 
dependability,  all  supported  the  general  conclusion  that  programmed 
instruction  can  be  at  least  as  effective  as  the  traditional  methods 
in  teaching  psychology  to  college  students.  Table  VII,  page  93, 
upholds  Nauruan' s  report  in  which  he  concluded  that  a  psychology  program 
is  relevant  and  applicable  for  students  studying  educational  psychology 
at  the  college  level. ^ 

Programmed  instruction  in  other  sub ject  areas .  The  scarcity  of 
available  reports  did  not  necessitate  the  construction  of  separate 
tables  for  the  individual  subject  areas  included  in  this  section. 
Consequently,  Table  VIII,  page  94,  was  organized  to  include  all  those 
reports  which  did  not  involve  mathematics,  science,  English  and 
psychology.  The  tabulation  did  not  identify  any  highly  dependable, 
specific  findings.  In  general,  the  greater  portion  of  the  studies 
supported  programmed  instruction  as  being  at  least  as  effective  as  con¬ 
ventional  techniques.  This  apparent  efficacy  of  programmed  instruction 
was  further  illustrated  by  three  other  somewhat  differently  designed 
studies.  Brinkman  found  that  the  experimental  group  in  Grade  eight 

^Theodor  F.  Nauman,  "A  Laboratory  Experience  in  Programed  Learn¬ 
ing  for  Students  in  Educational  Psychology,"  The  Journal  _of  Programed 
Instruction ,  I ; 1 7 ,  1963. 
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receiving  programmed  instruction  in  space  relations  made  significant 
gains  in  geometry  relative  to  a  carefully  matched  control  group  both 
in  content  learning  and  in  the  transfer  of  training.  Definite  con¬ 
clusions  were  impossible  because  Brinkman's  control  group  received  no 
instruction  in  spacial  relations  at  all.2  Hatch  reported  that  pilots 
exposed  to  programmed  instruction  by  machine  improved  significantly 
in  their  knowledge  of  inflight  job  information  while  those  who  were 
not  exposed  to  this  type  of  information  did  not  improve.2  Ferster 
showed  that  German  could  be  taught  with  a  great  deal  of  success  by 
using  programmed  instruction.  His  conclusions  were  based  on  an  indirect 
comparison  of  the  experimental  group's  performance  to  the  achievement 
of  previous  groups  who  took  whole  semesters  to  accomplish  the  same 
amount  of  work.^ 


Programmed  Ins truction  Compared  with  Conventional  Materials 

The  diversity  of  subject  areas  and  achievement  levels  along  with 
the  lack  of  specificity  concerning  instructional  objectives  and  ability 


o 

Erwin  Henry  Brinkmann,  "Educability  in  Visualization  of  Objects 
in  Space:  A  Programmed  Instruction  Approach"  (unpublished  Doctoral 
dissertation,  The  University  of  Michigan,  Ann  Arbor,  1963),  p.  71. 

^Richard  S.  Hatch,  An  Evaluation  of  Effectiveness  of  a  Self  - 
Tutoring  Approach  Applied  to  Pilot  Training  (WADE  Technical  Report 
59-320;  Wright-Patterson  Air  Force  Base,  Ohio:  Aero  Medical  Laboratory, 
Wright  Air  Development  Center,  Air  Research  and  Development  Command, 
United  States  Air  Force,  1959),  pp.  10-13. 

^Charles  B.  Ferster,  and  Stanley  M.  Sapon,  "An  Application  of 
Recent  Developments  in  Psychology  to  Teaching  of  German,"  Harvard 
Educational  Review,  28:59,  January-December ,  1958. 
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levels  precluded  the  formation  and  evaluation  of  any  specific  conclu¬ 
sions.  However,  with  the  exception  of  only  one  study,  all  comparisons 
as  shown  by  Table  IX,  page  98,  indicated  that  programmed  instruction 
was  at  least  as  effective  as  and  in  some  cases  superior  to  the  conven¬ 
tional  materials.  Dahl  found  that  his  control  group  using  the  con¬ 
ventional  teaching  text  scored  significantly  higher  on  the  textbook 
final  than  the  experimental  group  using  programmed  instruction.  The 
experimental  group,  however,  scored  higher  on  the  programmed  material 
test . ^ 

The  Comparative  Effectiveness  of  Program  Characteristics 

This  part  of  the  study  was  devoted  to  the  analytical  and  evalu¬ 
ative  survey  of  experimental  reports  concerned  with  the  three  following 
aspects  of  programmed  instruction:  the  presentation,  the  response, 
and  the  confirmation  or  feedback. 

Textbook  presentation  versus  machine.  The  examination  of 
Table  X,  page  100,  revealed  that  the  textbook  presentation  of  programmed 
material  was  apparently  equal  in  effectiveness  to  machine  presentation 
of  the  same  material.  From  a  more  specific  point  of  view,  only  one 
conclusion  was  found  to  exist  at  the  high  level  of  dependability.  It 
implied  that  mathematical  concepts  at  the  college  level  could  be  taught 
just  as  effectively  by  using  either  the  textbook  or  the  machine  presen¬ 
tation  of  programmed  material. 

^Ernest  Dahl,  "Report  of  an  Experiment  with  Programmed  Instruction 
in  Trigonometry  Using  TEMAC"  (Minneapolis,  Minnesota.  Fridley  Senior  High 
School,  Independent  School  District,  No.  14,  1963)  ,  p.  4.  (Dittoed.) 
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Group  versus  individual  pacing  in  programmed  instruction.  An 
indication  that  perhaps  there  is  no  difference  between  the  effective¬ 
ness  of  grouped  and  individual  pacing  was  implied  by  Table  XI,  page  102. 
When  such  factors  as  content,  ability,  achievement  level  and  instruc¬ 
tional  objectives  were  considered,  no  finding  was  identified  as  being 
highly  dependable. 

Small  steps  versus  large  steps .  Table  XII,  page  103,  showed  a 
tendency  for  the  small  step  program  to  be  more  effective  than  the  large 
step  program.  No  high  dependability  findings  were  revealed  by  the 
table . 


Use  of  repetition  in  presentation.  Table  XIII,  page  104,  dis¬ 
closed  no  apparent  indication  as  to  whether  or  not  repetition  is  a 
necessary  characteristic  of  programmed  instruction.  The  comparatively 
equal  distribution  of  the  supporting  indices  suggested  that  perhaps 
specific  conditions  influence  the  need  for  repetition.  No  finding  of 
a  highly  dependable  nature  was  found  because  of  the  lack  of  specificity 
of  the  data  gathered.  The  possible  influence  of  the  prevailing  circum¬ 
stances  upon  the  need  for  repetition  was  emphasized  by  Ferster  who 
reported  results  contrary  to  those  of  Holland  and  Porter^  who  used  the 

.  B.  Ferster,  "The  Role  of  Review  Material  in  Continuous  Pro¬ 
gramming  with  Teaching  Machines"  (Bloomington:  Indiana  Medical  School, 
no  date),  p.  3.  (Mimeographed.) 

7 James  G.  Holland  and  Douglas  Porter,  "The  Influence  of  Repetition 
of  Incorrectly  Answered  Items  in  a  Teaching-Machine  Program,"  Journal  of 
Experimental  Analysis  of  Behavior ,  4:305,  October,  1961. 
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same  program  in  their  experiment. 

Logical  versus  random  sequence  of  item  presentation.  Every 
available  report  which  was  analyzed  in  this  study  supported  random 
and  logical  sequences  as  being  equally  effective  in  the  presentation 
of  programmed  instruction.  According  to  the  established  criteria 
none  of  the  specific  findings  implied  by  Table  XIV,  page  106,  was 
rated  as  highly  dependable. 

Branched  versus  1 inear  presentation  of  programmed  mater ial s . 
Branching  and  linear  presentation  of  programmed  materials  were  found 
to  be  equally  effective  in  every  case.  However,  no  specific  finding 
indicated  in  Table  XV,  page  107,  was  found  to  be  highly  dependable. 

Inductive  versus  deductive  approaches  in  program  presentation. 
Only  two  studies  were  found  in  which  the  inductive  method  was  com¬ 
pared  to  the  deductive.  In  teaching  facts,  terms,  relationships , 
principles  and  concepts  in  algebra  to  a  group  of  Grade  eight  students, 
Belcastro  found  the  verbal  deductive  approach  to  be  significantly 
superior  at  all  levels  of  ability. ^  Birch  and  Stuckless  found  both 
approaches  to  be  equally  effective  in  teaching  grammar  and  composition 


O 

Frank  P.  Belcastro,  "Programmed  Learning:  Relative  Effective¬ 
ness  of  Four  Techniques  of  Programming  the  Addition  and  Subtraction 
Axioms  of  Algebra."  (unpublished  Doctoral  dissertation,  The  University 
of  Pittsburgh,  Pittsburgh,  1961),  pp.  54-55. 
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g 

to  adolescent  deaf  students.  Neither  finding  was  rated  as  highly 
dependab le . 

Comparative  effectiveness  of  question  and  statement  methods  in 
presenting  programmed  materials .  The  two  available  experimental 
reports  both  stated  that  the  question  and  statement  forms  of  program 
presentation  were  equally  effective  in  teaching  science.  Herrick's 
investigation  revealed  no  significant  difference  in  performance  between 
junior  high  school  groups  using  problem-setting  questions  and  those 
using  the  statement  form  in  presenting  the  same  content  material. ^ 

At  the  elementary  level,  McNeil  and  Keislar  reported  that  the  presen¬ 
tation  of  a  program  in  question  form  was  just  as  effective  as  the 
presentation  in  statement  form  for  all  characteristics  of  the  learner . ^ 
According  to  the  criteria  used  in  this  study  none  of  the  possible 
specific  conclusions  could  be  considered  highly  dependable. 


Jack  W.  Birch  and  E.  Ross  Stuckless,  "Programed  Instruction 
and  the  Correction  of  Written  Language  of  Adolescent  Deaf  Students" 

(a  research  report,  Special  Education  and  Rehabilitation,  School  of 
Education,  University  of  Pittsburgh,  Pittsburgh,  1963),  p.  45. 
(Mimeographed . ) 

■^Merlyn  Charles  Herrick,  "The  Effect  of  Problem-Setting  Ques¬ 
tions  on  Rate  and  Amount  of  Learning  in  Programming  Teaching  Machines" 
(unpublished  Doctoral  dissertation,  Indiana  University,  Bloomington, 
1962) ,  p.  31. 


^ ‘'’John  D.  McNeil  and  Evan  R.  Keislar,  "Individual  Differences 
and  Effectiveness  of  Auto-Instruction  at  the  Primary  Grade  Level," 
California  Journal  of  Educational  Research,  12:162,  September,  1961. 
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Comb ined  teacher  and  program  presentation  versus  program 
presentation  alone .  Ellson  et  a_l.  conducted  a  study  in  which  the 
effectiveness  of  classroom  and  program  procedures  was  compared  to 
that  of  a  program  procedure  alone.  They  found  that  a  group  of 
retardates  made  significantly  greater  gains  in  vocabulary  when  the 
combined  approach  was  used.  The  two  methods  were  equally  effective 
with  respect  to  comprehension.  Goldbeck  e_t  al.  used  the  combined 
approach  in  teaching  social  science  to  a  group  of  college  preparatory 
students.  They  reported  that  the  combined  approach  was  significantly 
more  effective  than  the  program  presentation  alone. 13  Although 
there  was  an  indication  that  perhaps  the  combined  approach  to  the 
presentation  of  learning  material  was  superior  to  the  program  presen¬ 
tation  alone,  no  highly  dependable,  specific  finding  was  identified. 

Selective  reference  versus  complete  presentation.  Bivens, 
Campbell  and  Terry  performed  a  study  in  which  they  found  that  self¬ 
directing  and  self -evaluating  programs  were  just  as  efficient  and 
effective  as  the  conventional  linear  programs  in  teaching  grade  nine 


1 2 

D.  G.  Ellson  e_t  a_ 1.  ,  "Comparison  of  Methods  of  Combining 
Classroom  and  Program  Procedures,"  Programmed  Teaching  of  E lementary 
Reading  --  a  Progress  Report ,  Experiment  V  (Bloomington:  University 
of  Indiana,  1962),  p.  11. 

13 

Robert  A.  Goldbeck  et  a_l.  ,  Integrating  Programed  Instruction 
with  Conventional  Classroom  Teaching.  AIR-C49-12/62-FR,  Final  Report 
(San  Mateo:  American  Institute  for  Research,  December,  1962),  p.  17. 


no 


14 

mathematics  to  a  group  of  low  ability  students.  Wahler  found  that  a 
diagnosis  of  difficulty  and  selected  reference  to  specific  frames  of 
a  program  produced  better  results  than  just  a  complete  presentation 
of  the  entire  program.  His  experiment  involved  teaching  punctuation 
in  grammar  to  a  group  of  average  and  below  average  grade  ten  students. 15 
Neither  finding  was  rated  highly  dependable. 

Constructed  response  versus  non- cons true  ted  response .  Tables  XVI, 
XVII  and  XVIII  showed  that  the  non-cons tructed  response  was  generally  as 
effective  as  the  constructed  response  with  some  tendency  for  the  con¬ 
structed  response  to  be  superior.  No  highly  dependable  findings  were 
identified.  Lambert,  Miller  and  Wiley  reported  a  study  in  which  they 
compared  the  two  modes  of  response  in  mathematics  at  the  high  school 
level.  Their  findings,  although  not  highly  dependable,  supported  the 
indication  that  the  two  response  modes  were  equally  effective.-*-^ 

Response  versus  no  response  in  progr ammed  instruction.  As  shown 
by  Table  XIX,  page  115,  most  studies  supported  the  finding  of  no 

^Lyle  W.  Bivens,  Vincent  N.  Campbell  and  Donald  F.  Terry,  Self- 
Direction  in  Programed  Instruction:  Effects  on  Learning  in  Low  Ability 
Students ,  AIR-DIO-7/63-TR,  Technical  Report  (Palo  Alto:  American 
Institute  for  Research,  July,  1963),  p.  7. 

"^Theodore  G.  Wahler,  "A  Selective  Reference  System,"  NSPI 
Journal,  11:14,  November,  1963. 

■^Philip  Lambert,  Donald  M.  Miller  and  David  E.  Wiley,  "Experi¬ 
mental  Folklore  and  Experimentation:  The  Study  of  Programmed  Learning 
in  the  Wauwatosa  Public  Schools,"  Journal  of  Educational  Research, 

55:491,  June-July,  1962. 
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difference  in  effectiveness  between  a  program  requiring  a  response 
and  one  which  does  not.  This  seemed  inconsistent  with  the  findings 
of  the  previous  section  in  which  a  strong  tendency  to  favor  construc¬ 
ted  response  programs  was  revealed.  None  of  the  specific  findings 
rated  high  in  dependability. 

Delayed  versus  immediate  confirmation.  No  specific  finding 
contained  in  Table  XX,  page  118,  was  rated  as  highly  dependable. 

From  the  overall  point  of  view,  it  was  found  that  the  indices  sup¬ 
ported  the  conclusion  that  immediate  confirmation  was  as  good  or 
better  than  delayed  confirmation. 

Type  of  confirmation.  Alter,  Eigen  and  King  found  that  con¬ 
firmation  alone  was  just  as  reinforcing  or  effective  as  confirmation 

and  trinkets  in  teaching  quantitative  concepts  and  numbers  to  five  and 

1  7 

six  year  old  children  representing  a  heterogeneous  ability  group. 

Melaragno  tested  the  hypothesis  that  some  negative  reinforcement  in 

an  autoins tructional  setting  does  not  hinder  learning  at  the  junior 

1  ft 

college  level.  °  He  found  that  massed  negative  reinforcement  was  less 
effective  than  spaced  negative  and  positive  reinforcement.  Although 
the  findings  were  not  judged  to  be  highly  dependable,  they  indicated 

^Millicent  Alter,  Lewis  Eigen  and  Shirley  King,  The  Effective¬ 
ness  of  Confirmation  Plus  Trinket  Reinforcers  in  Young  Chi ldren  (New 
York:  The  Center  for  Programed  Instruction,  No  date),  p.  4. 

1  8 

Ralph  J.  Melaragno,  "Effect  of  Negative  Reinforcement  in  an 
Automated  Teaching  Setting,"  Psychological  Reports,  7:384,  July- 
December,  1960. 
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that  perhaps  conf irmat ion  was  not  a  particularly  important  aspect  of 
programmed  instruction. 

Feedback  versus  no  feedback  in  programmed  instruction.  Table  XXI, 
page  120,  showed  that  programs  without  a  built-in  provision  for  feed¬ 
back  were  just  as  effective  in  most  cases  as  the  programs  with  the 
provision  for  feedback.  Only  one  specific  finding  was  rated  as  highly 
dependable.  It  implied  that  concepts  in  college  psychology  can  be 
taught  just  as  effectively  by  programs  which  do  not  utilize  a  provision 
for  feedback.  The  need  for  feedback  was  demonstrated  by  a  study  con¬ 
ducted  by  Stolurow  and  Lippert  who  found  that  confirmation  was  more 
effective  when  a  high  level  of  overlearning  was  required. 19  when,  how¬ 
ever,  a  low  level  of  learning  was  the  objective,  the  ordinary  prompting 
sequences  were  more  effective  in  producing  rapid  learning.  Since 
Stolurow' s  and  Lippert' s  subjects  were  educable  handicapped  children, 
one  cannot  generalize  in  interpreting  the  results  of  their  findings. 

Effect  of  ability  on  achievement  in  programmed  instruction.  A 
larger  portion  representing  a  greater  number  of  the  more  reliable 
studies  supported  the  finding  that  programmed  instruction  does  not 
reduce  the  dependency  of  learning  upon  ability.  Among  the  specific 
findings  included  in  Table  XXII,  page  122,  only  two  were  rated  as  highly 
dependable.  These  indicated  that  the  dependence  upon  ability  to  learn 

1 ^Lawrence  M.  Stolurow  and  Henry  Lippert,  Prompting,  Confirmation 
and  Vanishing  in  Teaching  of  a  Sight  Vocabulary  (Urbana:  Institute  for 
Research  on  Exceptional  Children,  University  of  Illinois,  April,  1962), 
p.  21. 
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concepts  and  principles  in  mathematics  at  the  elementary  and  the 
junior  high  school  level  was  not  reduced  by  programmed  instruction. 
Conflicting  evidence  did  not  permit  specific  findings  in  other 
subject  areas  and  at  other  ability  levels  to  be  rated  as  highly 
dependable . 


III.  COMMENTS 

Generally,  the  major  portion  of  the  findings  supported  pro¬ 
grammed  instruction  as  being  at  least  as  effective  as  the  conventional 
methods  in  producing  learning.  However,  it  was  found  that  an  appli¬ 
cation  of  an  evaluative  criteria  to  specific  situations  involving 
instructional  objectives  and  the  characteristics  of  the  learner  did 
not  reveal  as  many  dependable  findings  as  one  might  have  expected  from 
the  general  trend.  This  relatively  small  portion  of  dependable  findings 
was  largely  attributed  to  the  grossness  of  the  research.  The  analysis 
revealed  a  particularly  great  need  for  more  research  of  a  very  specific 
nature  in  new  subject  areas  as  well  as  in  those  in  which  some  investi¬ 
gation  has  already  been  made. 

This  criticism  of  grossness  applied  also  to  research  concerning 
the  nature  of  the  programs  and  their  relationship  to  effectiveness  of 
instruction.  The  study  showed  that  very  little  has  actually  been 
accomplished  in  determining  the  relationship  between  the  specific 
characteristics  of  the  program  and  those  of  the  instructional  objectives 


and  the  learner. 


CHAPTER  III 


SUMMARY  AND  CONCLUSIONS 

The  past  decade  has  seen  the  emergence  and  rapid  development 
of  what  some  educators  refer  to  as  the  "major  break  through"  in 
education.  Because  the  pace  of  research  in  programmed  instruction 
has  been  unusually  rapid,  it  was  decided  that  an  intensive  evaluative 
survey  of  the  experimental  work  in  this  area  would  be  very  beneficial 
in  revealing  more  vividly  the  potentialities  of  the  method.  Briefly, 
the  problem  was  to  find  and  evaluate  the  answers  to  a  number  of 
frequently  asked  questions  concerning  the  field  of  programmed  instruc¬ 
tion  . 


I.  THE  PROCEDURES  USED 

To  achieve  the  purpose  of  the  study,  the  available  research 
reports  were  evaluated  according  to  a  set  of  eight  criterion  items 
considered  to  be  an  adequate  representation  of  the  elements  of  accept¬ 
able  research  procedure.  The  evaluation  consisted  principally  of 
assigning  to  each  report  a  reliance  index  which  indicated  the  number 
of  criterion  items  that  it  satisfied.  These  reliance  indices  were 
subsequently  utilized  as  an  expression  of  supporting  power  in  deter¬ 
mining  the  dependability  of  the  answers  to  the  questions  posed  in  the 
study.  The  data  and  the  findings  obtained  from  the  analysis  of  the 
individual  reports  were  tabulated  in  order  to  identify  the  answers  to 
the  questions  and  to  determine  the  strength  of  the  support  they  received 
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in  terms  of  the  indices  derived  from  the  previous  evaluation  of  the 
reports.  The  strength  of  the  support  or  the  dependability  level  as 
it  was  referred  to  throughout  the  study  was  determined  by  dividing 
the  sum  of  the  supporting  indices  by  the  total  of  all  indices  pertinent 
to  the  specific  findings  involved.  The  findings  which  served  as  answers 
to  the  questions  were  considered  highly  dependable  when  the  ratio  of 
the  supporting  indices  to  the  total  indices  involved  resulted  in  0.9 
or  higher.  The  minimum  supporting  indices  were  set  at  8,  a  number 
which  represented  the  reliance  index  of  a  study  or  report  that  satis¬ 
fied  every  item  in  the  list  of  evaluative  criteria.  In  general,  the 
design  of  this  documentary  survey  was  a  major  modification  of  that 
used  by  Marrone  in  his  analysis  of  scientific  research  in  phonics.^ 

The  purpose  of  the  modification  was  to  increase  the  objectivity  of  the 
study  without  unduly  discrediting  the  nature  of  the  investigations  used 
as  sources  of  data  in  this  study. 

II.  FINDINGS  AND  CONCLUSIONS 

The  Evaluation  of  Individual  Reports 

Figure  9,  page  129,  shows  the  extent  to  which  each  criterion 
item  was  satisfied  by  all  the  reports  analyzed  in  this  study.  Explana¬ 
tion  of  terms,  the  error  rate  of  programs  and  the  reliability  of  the 
instruments  used  were  found  to  be  the  most  neglected  items  of  the 

^Victor  Eugene  Marrone,  "A  Critical  Analysis  of  Scientific 
Research  in  Phonics"  (unpublished  Doctoral  dissertation,  University  of 
Pittsburgh,  Pittsburgh,  1958),  pp.  17-18. 
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FIGURE  9 

PERCENTAGE  OF  STUDIES  SATISFYING 
EACH  OF  THE  CRITERION  ITEMS 
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criterion  list.  The  criterion  items  most  adequately  represented  were 
those  concerning  the  equating  of  groups  and  the  statistical  analysis 
of  data. 

The  Identification  and  Evaluation  of  the  Specific  Findings  Pertinent 
to  the  Delimiting  Questions 

Of  the  many  specific  findings  included  in  Tables  I  to  XXII  of 
Chapter  II,  only  those  rated  as  highly  dependable  received  emphasis  in 
this  section.  The  conclusions,  in  most  cases,  were  expressed  in  terms 
of  content  area,  achievement  and  ability  levels  of  the  learners 
involved  and  the  instructional  objectives  which  included  memorization, 
understanding,  application  of  the  acquired  learnings,  learning  of  facts 
and  concepts,  and  transfer.  The  decision  to  be  specific  in  identifying 
and  evaluating  the  findings  was  based  on  the  rationale  that  maximum 
utility  could  be  achieved  by  minimizing  reference  to  generalities. 

In  situations  where  some  reports  did  not  include  the  specific  data 
required,  this  technique  did  not  prove  to  be  entirely  successful.  How¬ 
ever,  in  spite  of  this  shortcoming,  the  method  was  found  to  be  quite 
effective  in  reducing  deceptive  apparencies. 

Programmed  instruction  in  exceptional  cases .  Every  available 
study  found  programmed  instruction  to  be  at  least  as  effective  as 
ordinary  procedures.  The  detailed  analysis  and  evaluation  revealed 
one  highly  dependable  finding  which  indicated  that  programmed  instruc¬ 
tion  was  at  least  as  effective  as  ordinary  methods  in  teaching  the 
recognition  of  words  in  spelling  or  reading  to  mentally  retarded  pupils. 
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Programmed  instruction  versus  conventional  techniques  in 
various  sub ject  areas  and  at  different  ability  and  achievement  levels . 
This  section  attempted  to  provide  answers  to  questions  concerned  with 
the  subject  areas  and  the  ability  and  achievement  levels  at  which 
programmed  instruction  could  be  used  successfully.  Since  the  success 
of  programmed  instruction  was  determined  on  the  basis  of  what  con¬ 
ventional  instruction  could  do  in  similar  circumstances,  the  method 
also  provided  some  answers  as  to  which  is  the  more  effective  of  the 
two  methods. 

In  mathematics,  at  the  elementary  level,  there  was  a  tendency 
to  favor  programmed  instruction  as  the  more  effective  method  of 
instruction.  The  conclusion  with  the  high  dependability  level  implied 
that  the  two  methods  were  equally  effective  in  teaching  problem  solving. 

At  the  junior  high  school  level,  a  greater  tendency  to  support 
programmed  instruction  as  the  superior  method  of  teaching  mathematics 
was  noted.  The  highly  dependable  finding  at  this  achievement  level 
revealed  the  two  methods  to  be  equally  effective  in  attaining  unspeci¬ 
fied  objectives  with  mixed  ability  groups. 

No  highly  dependable  findings  were  identified  concerning  mathe¬ 
matics  at  the  senior  high  school  level.  The  studies,  in  general, 
tended  to  support  the  two  techniques  as  being  equal. 

The  most  successful  utilization  of  programmed  instruction  in 
mathematics  seemed  to  occur  at  the  college  level.  Here,  the  findings 
indicated  that  the  two  methods  were  equally  effective  in  teaching 
concepts,  understandings,  application  of  the  acquired  knowledge,  and 
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transfer . 

In  English  at  all  levels  of  ability  and  achievement,  programmed 
instruction  was,  in  most  cases,  found  to  be  as  effective  as  the 
traditional  methods.  A  number  of  high  level  findings  were  identified 
within  this  subject  area.  The  two  methods  were  equally  effective  in 
teaching  concepts  to  high  ability  groups  and,  facts,  concepts  and 
unspecified  learning  material  to  average  ability  groups  at  the  elemen¬ 
tary  level.  At  the  junior  high  school  level  the  conventional  methods 
were  found  to  be  equally  effective  to  programmed  instruction  in 
teaching  understandings  and  other  unspecified  learning  material  to 
heterogeneous  ability  groups.  No  difference  in  effectiveness  between 
the  two  methods  was  found  in  attaining  unspecified  learning  objectives 
with  the  average  and  below  average  ability  groups  at  the  senior  high 
school  level;  however,  programmed  instruction  proved  to  be  more 
effective  with  the  high  ability  groups.  No  difference  in  effective¬ 
ness  to  teach  unspecified  learning  material  in  English  at  the  college 
level  was  found. 

In  other  subject  areas,  the  general  trend  supported  the  majority 
of  findings  in  English  and  mathematics  which  showed  that  the  two 
methods  of  instruction  were  equally  effective.  No  highly  dependable 
findings  were  identified  in  other  content  areas  such  as  psychology, 
social  studies  and  science. 

Programmed  instruction  versus  conventional  materials .  All  com¬ 
parisons  indicated  that  programmed  instruction  was  at  least  as  effective 
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as,  and  in  some  cases  superior  to,  other  instructional  materials. 

None  of  the  specific  findings  rated  highly  in  dependability.  The 
conventional  materials  were  represented  by  standard  textbooks  in  most 
cases . 

The  Comparative  Effectiveness  of  Program  Characteristics 

Numerous  tendencies  were  noted  concerning  the  methods  of 
presentation,  response  and  confirmation  or  feedback  in  programmed 
learning.  With  respect  to  the  presentation  of  programmed  materials 
it  appeared  that: 

1.  The  textbook  presentation  was  equally  effective  to  the 
machine  presentation  of  the  same  material. 

2.  The  individual  and  group  pacing  were  equally  effective. 

3.  The  small  step  was  superior  to  the  larger  step. 

4.  Repetition  did  not  always  add  to  the  effectiveness  of  the 
program. 

5.  There  was  no  difference  in  effectiveness  between  logical 
and  random  presentation  of  programmed  materials. 

6.  Branched  programs  were  equally  effective  to  the  linear 

type. 

7.  There  was  no  difference  in  effectiveness  between  the 
inductive  and  the  deductive  approaches  in  presenting  programmed  material. 

8.  Programs  in  question  form  were  equally  effective  to  programs 
in  statement  form. 

9.  There  seemed  to  be  some  advantage  in  combining  the  teacher 


and  program  methods. 
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The  analysis  of  research  reports  concerned  with  the  response  in  pro¬ 
grammed  instruction  indicated  that  the  non-constructed  response  was 
generally  as  effective  as  the  constructed  response  with  some  tendency 
for  the  constructed  response  to  be  superior,  and  that  in  many  cases 
a  program  requiring  no  response  was  just  as  effective  as  one  which 
requires  some  form  of  response.  With  regard  to  confirmation  or  feed¬ 
back  it  was  noted  that  little  difference  in  favor  of  immediate  con¬ 
firmation  existed  between  the  effectiveness  of  immediate  and  delayed 
confirmation.  Further  observation  indicated  that  in  the  majority  of 
cases  where  programs  with  a  provision  for  feedback  were  compared  to 
those  without,  there  was  no  difference  in  effectiveness.  It  was  also 
noted  that  the  type  of  confirmation  seemed  to  have  little  relation  to 
the  effectiveness  of  the  program.  Application  of  the  dependability 
formula  to  the  indices  supporting  the  specific  findings  showed  the 
following  to  be  highly  dependable:  textbook  and  machine  presentations 
of  programmed  materials  in  college  mathematics  were  equally  effective 
in  teaching  concepts;  and  programs  with  a  provision  for  feedback  were 
equally  effective  to  those  without  an  arrangement  for  feedback  in 
teaching  concepts  in  psychology  at  the  college  level. 

Achievement  in  Programmed  Instruction  as  a  Function  of  Ability 

Although  the  individual  studies  were  almost  equally  divided  in 
their  support  of  programmed  instruction  as  a  potential  for  reducing 
the  learner's  dependence  upon  his  ability,  the  highly  dependable  findings 
supported  the  conclusion  that  programmed  instruction  had  no  such  power 
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when  compared  to  conventional  teaching.  These  findings  indicated 
that  the  dependence  upon  ability  to  learn  concepts  and  principles  in 
mathematics  at  the  elementary  and  the  junior  high  school  level  was 
not  reduced  by  programmed  instruction. 

III.  DISCUSSION 

The  very  small  number  of  highly  dependable  findings  revealed 
by  the  application  of  the  evaluative  criteria  supported  the  general 
conclusion  that  programmed  instruction  was  at  least  as  effective  as 
conventional  methods.  It  seems  highly  probable  that  any  further  com¬ 
parison  of  conventional  and  programmed  instruction  will  serve  to 
produce  a  greater  number  of  highly  dependable  findings  which  will 
support  the  general  conclusion  reached  in  this  study.  The  greatest 
weakness  in  the  evidence  was  found  to  be  the  lack  of  important  details 
Many  writers  neglected  to  describe  either  the  program  or  the  objec¬ 
tives  to  be  achieved.  In  a  number  of  cases  the  purpose  of  the  instruc 
tion  was  stated  as  being  the  acquisition  of  knowledge  with  no  attempt 
made  to  elaborate  on  the  meaning  of  the  term  and  its  relation  to  the 
experiment.  There  was  no  precision  in  the  definition  of  "conventional 
instruction."  In  most  cases  it  represented  some  nebulous  method  which 
involved  a  live  instructor.  It  was  generally  assumed  that  a  small 
number  of  two  or  three  instructors  was  an  adequate  representation  of 
the  traditional  method.  Furthermore,  the  teachers  used  in  the  experi¬ 
ments  were  selected  on  the  basis  of  being  highly  competent.  The  fact 
that  all  the  teachers  involved  in  the  experiments  were  very  much  aware 
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that  they  were  being  used  in  evaluating  programs  could  also  have 
influenced  the  results.  In  numerous  cases  there  was  no  attempt  made 
to  separate  the  subjects  into  specific  ability  groups;  as  a  result, 
it  was  very  difficult  to  determine  which  type  of  student  could  benefit, 
most  from  programmed  instruction.  In  concluding,  it  may  be  said  that 
even  though  programmed  instruction  was  found  to  be  as  effective  as 
the  conventional  techniques,  much  remains  uncertain  with  respect  to 
the  specific  learning  situations. 

Since  very  little  has  been  established  concerning  the  charac¬ 
teristics  of  programmed  instruction  and  their  effect  upon  the  learning 
process,  it  may  be  inferred  that  the  meaning  of  programmed  instruction 
has  been  fixed  by  definition  only  and  not  by  its  effectiveness  to 
instruct.  Specifically,  the  effectiveness  of  programmed  instruction 
does  not  seem  to  depend  upon  the  nature  of  the  factors  implied  in 
the  usual  definition  of  the  term,  namely:  presentation,  response  and 
confirmation  or  feedback.  This  suggests  that  programmed  instruction 
could  best  be  defined,  at  least  for  the  moment,  as  any  method  found  to 
be  effective  in  producing  learning  and  which  does  not  directly  involve 
the  teacher. 

Perhaps  the  chief  advantage  of  programmed  instruction  over  any 
other  type  of  instruction  is  its  rather  impressive  superiority  in 
efficiency.  Every  study  which  reported  a  comparison  of  efficiency 
found  programmed  instruction  to  be  more  efficient  than  the  conventional 
methods.  This  feature  alone  would  justify  greater  use  were  it  not  for 
the  problem  of  administration  which  would  involve  such  matters  as 
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orientation  of  present  teaching  staffs,  training  of  new  teachers, 
selection  and  production  of  programs,  providing  new  facilities,  and 
modifying  promotion  policies.  As  a  result,  any  future  developments 
in  the  utilization  of  programmed  instruction  will  probably  take  place 
at  the  local  level  where  the  danger  of  becoming  too  involved  will  not 
be  as  great.  Unfortunately  there  is  a  danger  at  this  level  of  pro¬ 
grammed  instruction  becoming  a  victim  of  slip  shod  experimentation 
which  could  act  as  a  deterrent  in  any  possible  and  widespread  adoption 
of  its  use.  To  avoid  this,  it  would  be  desirable  for  such  institutions 
as  the  University,  the  Department  of  Education  and  the  Alberta  Teachers' 
Association  to  direct  certain  pilot  projects  in  an  effort  to  investi¬ 
gate  the  feasibility  of  encouraging  the  use  of  programmed  instruction 
in  our  schools. 


IV.  RECOMMENDATIONS  FOR  FURTHER  RESEARCH 

The  first  recommendation  is  that  many  of  the  previous  experiments 
be  replicated  with  emphasis  on  specific  things.  It  has  been  shown  by 
this  survey  that  gross  experiments  reveal  interesting  tendencies  which 
are  severely  limited  when  reference  to  them  is  made  in  dealing  with 
specific  issues.  With  respect  to  new  research,  Figures  1  to  8  of  the 
previous  chapter  are  vivid  indications  of  the  different  types  of  experi¬ 
ments  which  could  be  recommended  for  the  purpose  of  determining  the 
feasibility  of  programmed  instruction  in  various  other  content  areas. 

For  instance,  Figure  2,  page  77,  shows  that  very  little  experimentation 
has  been  done  in  programmed  mathematics  with  average  and  low  ability 
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groups  at  the  junior  high  school  level.  Similarly,  the  figures 
reveal  that  the  problem  of  transfer  has  received  very  little  atten¬ 
tion  compared  to  other  instructional  objectives. 

It  is  further  recommended  that  the  principle  of  specificity  be 
carefully  adhered  to  in  performing  experiments  concerning  the  charac¬ 
teristics  of  programmed  instruction.  For  example,  it  may  be  that 
the  effectiveness  of  a  program  characteristic  is  dependent  upon  the 
learner's  ability  and  the  type  of  instructional  objective  to  be 
achieved.  Gross  experiments  are  not  likely  to  disclose  these  rela¬ 
tionships  . 

The  final  recommendation  concerns  the  evaluation  of  programs. 

Many  experiments  involve  two  processes,  namely:  the  construction  and 
the  evaluation  of  a  certain  program.  In  most  cases  this  is  done  by 
comparing  the  effectiveness  of  the  program  with  that  of  the  instruction 
given  by  one  or  a  small  group  of  teachers  to  a  classroom  of  students. 
Such  a  design  has  two  glaring  weaknesses:  (1)  a  small  group  of  teachers 
does  not  represent  a  typical  teacher,  and  (2)  in  a  classroom  situation 
there  is  no  ideal  teacher- to-student  relationship  so  that  in  effect 
the  findings  are  actually  the  results  of  a  comparison  between  the 
effects  produced  by  group  and  individual  treatments.  Briefly,  it  is 
the  effectiveness  of  the  circumstances  surrounding  the  instructional 
procedure  that  is  being  investigated  and  not  the  instruction  itself, 

A  more  appropriate  and  accurate  evaluation  would  involve,  a  large  sample 
of  teachers  adequately  representing  a  particular  teaching  force.  Each 
teacher  from  this  sample  would  then  be  randomly  assigned  to  one  member 
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of  a  homogeneous  group  of  learners  representing  the  control  subjects „ 
A  similar  group  of  experimental  subjects  would  be  given  the  program 
treatment.  It  is  felt  that  the  results  produced  by  this  design 
could  more  justifiably  be  considered  an  evaluation  of  programmed 
instruction. 
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